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Executive Summary. 


What distinguishes high schools that “beat the odds” for students from traditionally lower-performing 
groups from schools that struggle to improve the achievement and graduation rates of these student 
populations? What types of programs, practices, and processes support better than expected outcomes for 
students at risk of failure? How can districts identify, adapt, and scale up these practices to their less 
effective high schools? 

These are the questions that the National Center on Scaling Up Effective Schools (NCSU) is addressing. 
NCSU — a collaborative partnership between research universities, developers, and two large urban 
districts — is a five-year project funded by the Institute of Education Sciences in the U.S. Department of 
Education. NCSU focuses on identifying the combination of essential components and the programs, 
practices, processes, and policies that explain why some high schools in large urban districts are 
particularly effective at serving low-income students, minority students, and English language learners. 
We then work collaboratively with the districts to develop processes to share and implement these 
practices in less effective high schools. 

This report presents findings from the first phase of this work — the identification of practices that 
distinguish higher and lower value-added high schools in one of our partner districts, the Fort Worth 
Independent School District (FWISD). The findings from this report will be used to define a “design 
challenge” that will guide a collaborative design process that will develop an innovation to be 
implemented in the district. 

Research Design 

This data herein come from a comparative case study of four high schools in FWISD during the 2011- 
2012 school year. The study was designed to identify the programs, policies, and practices that effective 
schools in FWISD used to coordinate successful outcomes for students. The four schools were selected 
based on school-level, value-added student achievement measures. The value-added measures of 
achievement were created in reading, mathematics, and science for all students in the school, and for 
subgroups of students by race/ethnicity, free and reduced lunch eligibility status, and English language 
learner (ELL) status. In short, two schools were selected with relatively higher value-added results and 
two with relatively lower value-added results. In each school, we conducted approximately: 

• 9 focus groups (with students, teachers, and student activity leaders); 

• 50 interviews (with principals, assistant principals, teachers, guidance counselors, support 
personnel, and students); 

• 70 observations in English, mathematics, and science classrooms; and 

• 9 student shadowing observations. 

Data collection primarily focused on 9 th -and 10 th -grade students and teachers in English, mathematics, 
and science, although we balanced this focus with other data from key staff and a cross section of the 
school (e.g., teacher focus groups spanned all grades and subject areas) to gain a comprehensive 
understanding of our schools. In addition to this fieldwork, we collected numerous school ai t i facts (e.g.. 
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documents about the school or processes within the school such as the teacher handbook, academic 
profile, academic learning walk criteria, etc.) and analyzed administrative, disciplinary, and course-taking 
data from the district, as well as survey data from teachers, students, and parents. 

Main Findings 

Our analyses revealed that the practice of increasing student ownership and responsibility for their 
academic success emerged as a distinguishing feature of schools with higher value-added student 
achievement over those with lower value-added achievement. Increasing student ownership and 
responsibility for their academic success means creating a set of norms and school-wide practices that 
nurture a culture of learning and engagement among students. Students who are taking responsibility for 
their own learning are personally invested in their education and committed to understanding the work. 
Putting such a focus in place involves building students’ confidence and understanding of how they can 
take responsibility for their own academic success. We emphasize two activities important for increasing 
this capacity: 1) changing beliefs and mindsets of students to increase self-efficacy (that is, an 
individual’s beliefs about his or her ability to perform behaviors that should lead to expected outcomes) 
and 2) engaging students to do challenging academic work. 

Notably, the higher value-added schools neither assumed that students would develop this ownership on 
their own, nor merely declared it as an expectation. Rather, teachers and other adults scaffolded students’ 
learning of both academic and social behaviors and put structures in place to guide them in taking 
ownership and responsibility for their academic success. That is, these schools made a concerted effort to 
provide encouragement and support to students. Both of our higher value-added case study schools 
provided this scaffolding through integrated strategies of academic press and academic support for 
students. Furthermore, teachers adopted — and were held accountable for — the perspective that student 
ownership for learning is important and should be developed. These findings suggest that high schools 
can address gaps in student achievement and equip students to meet the educational challenges and 
workforce demands of the twenty-first century, by developing programs, processes, and practices that 
fully engage students and develop them as self-directed learners. 

The results further indicated that this emphasis on student ownership and responsibility was effective 
because it was enacted through what we call “school-wide facilitating conditions” that include: the 
development of a shared school mission; alignment of school-wide structures and practices to the 
mission; a culture of trust; faculty and student stability; the presence of caring and positive relationships 
between students and teachers; individual and collective teacher efficacy, teacher accountability, and a 
safe and orderly school environment. 

Next Steps 

The next stage of the Center’s work involves bringing district leaders, school leaders, and teachers 
together to collaborate in the design and implementation of an innovation to increase students’ ownership 
and responsibility for their own academic success to be used in other FWISD high schools. In this way, 
the central findings from this report will define a design challenge to guide a collaborative design process. 
A District Innovation Design Team (DIDT) will develop an innovation based on the research findings 
presented in this report, the broader research literature on effective practices, and a needs assessment on 



what aspects of student ownership and responsibility are currently in place in their high schools. Then 
School Innovation Design Teams (SIDTs) will pilot, adapt, and implement the innovation in three 
schools. As part of this process, the innovation schools themselves will also study and evaluate the impact 
with an eye to understanding the effort required to scale up the innovation. The researchers in the Center 
will then study and evaluate this implementation, examine its impact, and assess the district’ s ability to 
support and scale up the designed interventions to additional high schools. 
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Introduction. 


What distinguishes high schools that “beat the odds” for students from traditionally lower-performing 
groups from schools that struggle to improve the achievement and graduation rates of these student 
populations? What types of programs, practices, and processes support better than expected outcomes for 
students at risk of failure? How can districts identify, adapt, and scale up these practices to their less 
effective high schools? 

These are the questions that the National Center on Scaling Up Effective Schools (NCSU) is addressing. 
NCSU — a collaborative partnership between research universities, developers, and two large urban 
districts — is a five-year project funded by the Institute of Education Sciences in the U.S. Department of 
Education. NCSU focuses on identifying the combination of essential components and the programs, 
practices, processes, and policies that explain why some high schools in large urban districts are 
particularly effective serving low-income students, minority students, and English language learners. We 
then work collaboratively with the districts to develop processes to share and implement these practices in 
less effective high schools. 

NCSU works with high schools for four main reasons. First, the overwhelming majority of research on 
effective schools and school reform is limited to elementary schools. Secondary schools are larger, 
organizationally more complex, and politically more complicated with multiple administrative layers and 
subject-based teachers and other specialists that often create natural divisions among staff 1 and result in 
disagreements around goals, policies, and practices. Such factors make the process of change more 
difficult in secondary schools 11 . Second, national, and international comparisons of student achievement 
indicate that, despite progress in elementary grades, underperformance in high school is a persistent 
problem. 111 There are extraordinary economic and educational consequences for students who are neither 
college nor workforce ready. Third, national attention has focused on the need to prepare students to 
succeed in college and careers; high schools play a critical role in achieving these national goals. Finally, 
as prior research suggests, the relative influence of non-school factors, such as family background, on 
academic success decreases as students progress through school suggesting widespread opportunities for 
effective schools to make real and lasting change in student outcomes. IV Put simply, identifying practices 
that make high schools effective holds the promise of increasing the outcomes and life opportunities of 
students. 

The Center’s work includes several phases. The first involved intensive data collection in our partner 
district to identify practices that distinguish higher and lower value-added high schools. The second phase 
uses the central findings from the first phase to define a “design challenge” that will guide a collaborative 
design process in developing innovations based on these research findings. In the third phase, three lower 
value-added schools will adapt and then implement the designed innovation as we evaluate this 
implementation, assess the impact of the intervention designs using interrupted time series analysis with 
comparison schools, and assess the district’s ability to support and scale up the designed interventions to 
additional high schools. Involving partners from the participating district on the design team will take 
advantage of district expertise, help to insure that practices identified for “transfer” are aligned with 
district’s current goals and initiatives, and help to bring legitimacy to the transfer process. 



This report presents results from the first phase of our work with one of our partner districts, Fort Worth 
Independent School District (FWISD). V This phase of the Center’s work identified schools in the district 
that are more and less effective at improving student achievement in English/language arts, mathematics, 
and science through analyzing value-added student achievement data. We then used a combination of 
interviews, surveys, and observations to uncover practices and expectations in the higher value-added 
schools that contribute to their success and distinguish them from the lower value-added schools in the 
same district. 

Through our analyses, the practice of increasing student ownership and responsibility for their academic 
success emerged as a distinguishing feature of schools with higher value-added student achievement over 
those with lower value-added achievement. These findings suggest that high schools can address gaps in 
student achievement and equip students to meet the educational challenges and workforce demands of the 
twenty-first century, by developing programs, processes, and practices that fully engage students and 
develop them as self-directed learners. The results further indicate that this emphasis on student 
ownership and responsibility was effective because it was enacted through what we call “school-wide 
facilitating conditions” that include: the development of a shared school mission; alignment of school- 
wide structures and practices to the mission; a culture of trust; faculty and student stability; the presence 
of caring and positive relationships between students and teachers; individual and collective teacher 
efficacy, teacher accountability, and a safe and orderly school environment. 

The Center did not set out to identify student ownership and responsibility for academic success as the 
practice leading to greater-than-expected gains from students in traditionally low-performing groups. 
Rather, our data collection and analysis were organized around what we term the “eight essential 
components of effective schools:” 

• Learning-centered Leadership 

• Rigorous and Aligned Curriculum 

• Quality Instruction 

• Personalized Learning Connections 

• Culture of Learning and Professional Behavior 

• Connections to External Communities 

• Systemic Performance Accountability, and 

• Systemic Use of Data. 

The components of this framework are conceptualized as working together in effective high schools to 
create deep connections and relationships for both adults (leaders, teachers, and staff) and students. Our 
framework emphasizes that it is not the adoption of any individual component through specific programs 
or practices that leads to school effectiveness, but the integration and alignment of school processes and 
structures across these eight components .' 1 Although a consensus has recently begun to emerge around 
these components of successful schooling, far less is known about the ways in which educators develop, 
implement, integrate, and sustain these components. This is where the current report hopes to shed light. 

Our analysis of four FWISD case studies revealed that those schools with higher value-added student 
achievement held high expectations for the learning of all students and enacted practices that helped 
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students assume ownership and responsibility for their learning. Notably, schools neither assumed that 
students would develop this ownership on their own, nor merely declared it an expectation. Rather, 
teachers and other adults scaffolded students’ learning of both academic and social behaviors and put 
structures in place to guide students in taking ownership and responsibility for their academic success. 
That is, these schools made a concerted effort to provide encouragement and support to students. 
Furthermore, teachers adopted — and were held accountable for — the perspective that student ownership 
for learning is important and should be developed. The specific practices and school-wide conditions that 
we observed for facilitating this kind of academic environment will be described in more detail in later 
sections. 

The next stage of the Center’ s work involves district leaders and school leaders and teachers coming 
together to design and implement an innovation to increase academic press and students’ ownership and 
responsibility for their own academic success in other high schools in the district. This collaborative 
process will seek to develop an innovation (e.g., a school-level program or practice) that is focused on 
increasing student ownership and responsibility. Much of the work will be performed by a District 
Innovation Design Team (DIDT) (such as teachers, other school-level personnel, central office personnel, 
and researchers) and School Innovation Design Teams (SIDTs). Our colleagues at the Educational 
Development Center (EDC), a global non-profit based in Waltham, Massachusetts, who have experience 
developing leadership and instructional programs will facilitate this process, guiding the DIDT and SIDTs 
through the steps of 1) developing a prototype; 2) testing the ideas in the prototype; 3) learning from the 
testing and making revisions; and 4) adapting to school context based on the learning, prior to initial 
implementation in three district high schools. Involving partners from the participating district on the 
design team will take advantage of district expertise, help to ensure that practices identified for “transfer” 
are aligned with the district’s current goals and initiatives, and help to bring legitimacy to the transfer 
process. 

This report is divided into six sections. The first describes the state and district context for our work, as 
well as the data and methods used in this study. The second section provides brief summaries of the four 
schools that participated in this study. The third section presents the main findings, including the focus on 
increasing student responsibility and the facilitating conditions that further enabled school success. The 
fourth section describes the essential components that served as the theoretical framing for this study and 
summarizes findings across our four case study schools within these components. The fifth section 
presents seven specific examples of promising practices to illustrate how the study high schools 
effectively implemented practices to increase student ownership and responsibility. The final section 
provides concluding comments and outlines the next steps for the Center. 



Research Design. 


State and District Context 

Fort Worth Independent School District (FWISD) is the sixth largest district in Texas and the 39 th largest 
in the country, serving over 80,000 students. Table 1 presents 2010-11 student demographic 
characteristics and assessment results (based on the Texas Assessment of Knowledge and Skills, or 
TAKS) for the state (excluding charters) and FWISD. ™. During that period students in FWISD were 
predominantly Hispanic (59%), African American (23%), and economically disadvantaged (76%). Over a 
quarter were designated as Limited English Proficient (LEP). The achievement results demonstrated that 
FWISD lagged behind state averages, as is the case in most large urban districts. FWISD had 142 
campuses at that time, 14 of which were high schools. FWISD had been rated Academically Acceptable 
by the state for the past three years. 


Table 1. Outcome and Demographic Data for Texas and Fort Worth Independen t School District 


Variable 

State of Texas 

Fort Worth ISD 

Outcomes (all grades ) 

Percent passing Reading/ELA 

90 

82 

Percent passing Writing 

92 

86 

Percent passing Mathematics 

84 

75 

Percent passing Science 

83 

73 

Percent passing Social Studies 

95 

92 

Four-year graduation rate (Class of 2010) 

84 

79 

Attendance rate 

96 

95 

Average SAT score 

985 

903 

Average ACT score 

21 

18 

Demographics (all grades) 

Total students 

4,778,688 

81,511 

Percent African American 

13 

23 

Percent Hispanic 

50 

59 

Percent White 

32 

14 

Percent other race/ethnicity 

6 

3 

Percent economically disadvantaged 

59 

76 

Percent Limited English Proficient (LEP) 

17 

28 


Note: Data come from the Texas Education Agency (http://ritter.tea.state.tx.us/perfreport/snapshot/201 1/state. html) 


Texas has a long history of test-based accountability, beginning with the Texas Assessment of Basic 
Skills (TABS) in 1980 and the first school accountability ratings in 1993. The high-stakes accountability 
environment has only increased over time. During the 2011-12 school year when the data for this report 
were collected, the assessment, accountability, and graduation requirement programs were in transition. 
The first major change concerned the state assessment. The TAKS assessment used when this project 
began will be phased out completely after the class of 2014 graduates. TAKS was composed of end-of- 
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grade assessments in math, reading and/or ELA, writing, science, and social studies in grades 3-11, with 
graduation requirements tied to passing these exams. Beginning in spring 2011, Texas rolled out the State 
of Texas Assessments of Academic Readiness (STAAR), which will represent a significant departure 
from the TAKS in high schools. Specifically in 2011-12 STAAR included 12 end-of-course (EOC) 
assessments at the high school level, in place of the four graduation tests used under TAKS.™ 1 The EOCs 
will assess Algebra I, geometry, Algebra II, biology, chemistry, physics, English I, English II, English III, 
world geography, world history, and U.S. history. In addition, STAAR was designed to be more rigorous 
and measure greater depth of understanding. Changes were made to align the state assessment with 
postsecondary readiness measures. Further, STAAR has greater implications for students. Each class that 
culminates with an EOC assessment will count the score as 15% of the student’s final grade in the 
subject. Further, high school graduation is linked to a student’s cumulative score across 8-12 EOCs. 

The transition in assessments created two key challenges for high schools in Texas at the time of this 
study. For starters, first-time 9 th -graders faced different graduation requirements (focused on the STAAR 
EOC tests) than students in grades 10 to 12 (which were focused on the TAKS). Second, performance 
standards for the secondary level assessments were still being determined when we began data collection. 
Thus, teachers did not know what cut scores would be used to distinguish the three levels of academic 
performance — advanced, satisfactory, and unsatisfactory. Thus, while the transition to STAAR 
emphasized increased rigor, depth, and complexity, the absence of performance standards meant teachers 
did not have complete information about the STAAR. 

The transition in assessment was accompanied by an overhaul of the state accountability system. In 
addition to federal Adequate Yearly Progress (AYP) requirements, the state accountability system 
assigned the following ratings to schools: Exemplary, Recognized, Academically Acceptable, and 
Academically Unacceptable. For the 2009-2010 and 2010-201 1 school years, both federal adequate yearly 
progress (AYP) and state accountability ratings were based on the TAKS. However, with the transition to 
the STAAR, the state decided that no new state accountability ratings would be assigned based on state 
assessment results from the 2011-2012 school year. The state was still required to make annual federal 
AYP determinations, however the criteria for doing so were not available until after most testing had 
ended. Texas is developing a new school rating system and the first state accountability ratings based on 
the new STAAR assessments were released in Summer 2013. 

FWISD has responded to the Texas state standards and accountability system by developing detailed 
curriculum frameworks it expects to implement across the district. These frameworks were developed by 
teachers and include pacing guides and recommended activities for most grades and classes (particularly 
in tested subjects). The curriculum frameworks are reinforced by district-wide Curriculum-Based 
Assessments (CBAs) that are benchmark assessments designed to assess the extent to which students 
have mastered content that was represented on the curriculum frameworks for the preceding time period. 
Depending on the subject, CBAs are administered every six or eight weeks. 

In addition, Texas has established three different high school diploma pathways minimum high school 
program, recommended high school program, and distinguished achievement (advanced) high school 
program. The key difference is an increase in course requirements, particularly in mathematics, science, 
social studies, and foreign language. 



The national economic crisis also has affected schools in Texas. As happened in many other states, the 
crisis led to financial strictures in Texas that resulted in the elimination of billions in state appropriations 
for schools. State education spending has been frozen at 2006 levels, causing personnel concerns in 
FWISD, as payroll constitutes 84% of its budget. Thus both the state and district were operating under 
severe budget constraints at the time of this study. 

In the 2011-12 school year, FWISD was in the process of hiring a new superintendent, who was 
announced in January 2012 (in the midst of our data collection). The district has a number of programs in 
place, include site -based-decision-making, pay for performance, and themed school choice programs. 
Specifically, the FWISD site -based-decision-making (SBDM) program was approved in 1992 with a goal 
of improving student achievement by tapping the input of multiple stakeholders. Each SBDM team 
consists of the principal, four teachers, three parents, one campus-level nonteaching professional staff 
member, one district-level nonteaching professional staff member, two community members, and two 
business community members. Second, the district was in the first year of implementing a federal Teacher 
Incentive Fund grant known as RISE (Redesign to Increase Schools of Excellence). The goals of the 
program (which was being implemented in 19 schools) include rigorous teacher and principal evaluation, 
high expectations and achievement, changing school climates, and building teacher and principal 
capacity. The RISE program replaces a prior performance pay initiative called PEAK, which was in place 
from 2008-2011. 

Another program, the FWISD Gold Seal Programs of Choice program (GSPOC) was begun during the 
201 1-12 school year to allow students to choose a course of study based on their interests. All high 
schools offer GSPOC options, and three high schools have additional choice elements. Choice options 
also exist at the middle and elementary levels. The high school GSPOC program is being phased in one 
grade at a time, so it affected only 9th-graders in 2011-12. Students must apply even if the program is at 
the student’s zoned school. Program of Choice and School of Choice students receive priority in course 
placement. Despite the fact that this was the first year of implementation, students in grades 10, 11, and 
12 may also have been enrolled in courses affiliated with the GSPOC as many of these programs evolved 
from existing magnet programs or specialties in the school. Transportation is provided to FWISD students 
if they choose a program that is not available at their zoned, neighborhood school. 
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Data and Methods 


The data in this report come from a comparative case study of four high schools in FWISD during the 
2011-2012 school year. The schools were selected based on school-level value-added measures. In short, 
two schools had relatively higher value-added (HVA) results and two had relatively lower value-added 
(LVA) results. 

Data were collected in Fort Worth in three time periods during the 2011-12 school year. Data collection 
methods included focus groups (with students, teachers, student activity leaders, district parent liaisons); 
interviews with principals, assistant principals, guidance counselors, support personnel, teachers, 
students, district personnel, and students; observations of English, mathematics, and science classrooms; 
shadowing students during their regular school day; and the collection of school and district artifacts. 

Data collection primarily focused on 9 th - and 10 th -grade students and teachers in English, mathematics, 
and science, although we balanced this focus with other data from key staff and a cross-section of the 
school (e.g., teacher focus groups spanned all grades and subject areas) to gain a comprehensive 
understanding of our schools. Table 2 below shows the amount of data collected by school. See Appendix 
A for more information on the data and methods. 


Table 2. Types and Amounts of Data Collected at Case Study Schools 



LVA Schools 

HVA Schools 

Data type 

Mountainside 

Valley 

Lakeside 

Riverview 

Interviews 

51 

50 

54 

48 

School Administrators 

4 

5 

5 

7 

Teachers 

18 

19 

18 

18 

Deans of Instruction 

2 

2 

1 

0 

Department Heads/Lead Content Teachers 

6 

6 

3 

6 

Guidance Counselors 

2 

4 

3 

2 

Support Personnel 

9 

5 

16 

5 

Students 

10 

9 

8 

10 

Focus groups 

9 

8 

9 

9 

Students 

3 

3 

3 

3 

Teachers 

3 

3 

3 

3 

Student Activity Leaders 

3 

2 

3 

3 

Observations 

76 

81 

87 

78 

Classroom Periods 

66 

72 

73 

68 

Students Shadowed 

10 

9 

8 

10 

Faculty/School Administrative Team 
Meetings 

0 

0 

6 

0 


We observed and videotaped a total of 274 class periods of English Language Arts (ELA), math and 
science. The same teachers who participated in the interviews were also observed. Four class periods per 



teacher were videotaped and coded by trained observers. In most cases, two of the class periods were 
observed in Wave 1 and two were observed in Wave 2 (although a few observations that were missed due 
to teacher absences or scheduling difficulties were made up in Wave 3). We used an observational tool 
called the Classroom Assessment Scoring System - Secondary (CLASS-S) 1X to assess teacher-student 
interactions in the classroom. We observed and coded the following domains and dimensions using the 
CLASS-S framework: Emotional Support (positive climate, negative climate, teacher sensitivity, regard 
for adolescent behavior), Classroom Organization (behavior management, productivity, instructional 
learning formats), Instructional Support in the classroom (content understanding, analysis and problem 
solving, quality of feedback, and instructional dialogue), and Student Engagement. See Appendix B for 
additional information on the classroom observations. 

In our final wave of data collection in late spring 2012, we conducted 37 student shadowing observations 
in which a member of the research team accompanied a student through his or her school day. Every five 
minutes, the researcher recorded data in an electronic log detailing the class period, precise time, course 
subject, academic track, location, the teacher’s expectation of the student (what the student was supposed 
to be doing), the academic nature of that task (i.e., related to content or not), and level of student 
engagement in that task (active engagement, passive engagement, not engaged). If the student was off- 
task the observer noted what behavior the student in engaging in and with whom the student was 
interacting. The coding for activities in which teachers expected students to engage included: whole class 
discussion, direct instruction, pair or group work, individual work, an interactive or student led activity, 
taking a test or quiz, transitioning between activities, other academic activities (includes watching or 
giving a presentation, general studying, watching a film or video, or academically-oriented talk with the 
teacher), non-academic activities (such as socially-oriented talk with the teacher, handing out report cards, 
saying the Pledge of Allegiance, school announcements), other (for non-core subjects, this includes doing 
Reserve Officer Training Corps (ROTC) drills and playing sports in gym class), and nothing (i.e., there is 
nothing the student is supposed to be doing at that moment). See Appendix C for more information on the 
student shadowing observations. 

In addition to this fieldwork, we obtained administrative, disciplinary, and course-taking data from the 
district. We also collaborated with the district to obtain survey data from teachers and students. Analyses 
of administrative data are presented as school-level means or percentages. Analyses of survey data are 
presented as school-level means, alongside the district average across all high schools. Statistical 
significance was calculated based on mean comparisons tests between each case study school’s mean 
scale rating compared to the mean from the other 12 high schools in the district. Appendices D and E 
provide details on the student and teacher survey data, respectively. 
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Case Summaries. 


In this section, we describe the two lower value-added (LVA) schools and the two higher value-added 
(HVA) schools. Table 3 provides data on the demographic characteristics and value-added rankings. To 
protect the identity of the schools and the participants therein, we have provided ranges and used 
pseudonyms. We then provide brief case summaries focusing on the school context, cross-cutting themes, 
and main findings. Note that due to our sampling strategy, we refer to the schools as either lower value- 
added (LVA) or higher value-added (HVA). However, as described in the case summaries below, in some 
cases schools may be performing relatively better in some subject areas or for some student subgroups 
than others and thus one LVA school (Valley) has relatively strong outcomes in some indicators and one 
HVA school (Riverview) has relatively weak outcomes in some indicators. This continuum of 
performance outcomes is also evident in our findings related to student responsibility and ownership. 

Note that the case summaries do not focus on listing formal structures or programs in each school. Rather, 
we focus instead on how systems or practices were enacted, regardless of their formal structure. This is 
because the formal structures or systems in the schools were largely similar — such as decisions to have 
grade-level teaming, professional learning communities, common planning time within departments, 
common assessments, etc. It is likely that the district influence shaped the basic structure of these schools. 
When individual schools do have unique structural elements that appear to facilitate or impede school 
improvement, we do describe those elements. 


Table 3: Demographic Characteristics and Performance Indicators of Case Study High Schools 



LVA Schools 

HVA Schools 


Mountainside 

Valley 

Lakeside 

Riverview 

School characteristics 

Enrollment 

700-1200 

>1500 

700-1200 

>1500 

Percent Black 

>50% 

<20% 

<20% 

<20% 

Percent Hispanic 

<40% 

>75% 

>75% 

41-75% 

Percent economically 

60-75% 

>75% 

>75% 

<60% 

disadvantaged 

Percent Limited English 

<7% 

>7% 

>7% 

<7% 

Proficient 

2010 Graduation Rate 

<80% 

<80% 

>85% 

>85% 

2011 State Rating 

Academically 

Unacceptable 

Academically 

Acceptable 

Academically 

Acceptable 

Academically 

Acceptable 

Value-added rank within district, all 

subjects, all students (out of 13 total) 

13 

11 

1 

3 


Note: The state accountability rating and graduation rate were the most recent data available at the time of school 
selection. Demographics represent the composition of the schools at the time of our visits (201 1-12). The value- 
added ranks are derived from 3 years of data of school-level value-added in math, science, and reading. The most 
recent year was 2010-11. 



Mountainside High School 

One LVA school is Mountainside High School, which is predominantly African-American. The value- 
added indicators ranked it last of the 13 high schools in the district in reading and science, both for all 
students as well as the two largest subgroups: African-Americans and those eligible for free- and reduced- 
price lunches. School outcomes such as graduation rates, dropout rates, and state accountability ranking 
were also very low compared to district averages. 

Multiple participants with whom we interacted at Mountainside reported that while many elements of the 
school are functional, other systems that allowed the school to operate as an effective organization in the 
past may have been breaking down. Several participants asserted that systems such as discipline and 
scheduling did not function as they should and thus complicated the work of faculty. Multiple participants 
reported that a lack of communication and trust prevented staff members from working together to meet 
the needs of students. Some participants traced these breakdowns largely to instability caused by 
significant turnover in personnel. In addition, multiple participants reported a culture of “multiple 
chances” that allowed students to make up poor or missed work, while some students were not punished 
for disciplinary infractions. Although offering additional opportunities for success can provide some 
benefits, the practices at Mountainside appeared to reduce student accountability and allow students to 
take the easiest path to graduation. 

Valley High School 

Valley High School, which serves a predominantly Hispanic and economically disadvantaged student 
population, had undergone considerable change in the recent past. We describe Valley as a lower value- 
added school because the value-added measures indicate the school was near the bottom of the district 
when combining all subjects for all students. However, for some subgroups and subjects, the school was 
closer to the district average. The graduation and dropout rates were near the district median. 

A school culture that included gangs and low achievement rates led Valley to become a turnaround school 
several years ago, with a new principal who was given autonomy to hire staff. The principal hired a new 
administrative team and 40 percent of the faculty, educators who were willing to work long hours and 
emphasized outreach to students. Valley has since succeeded in the areas targeted for turnaround — 
creating a more positive school culture, building personalized relationships with students, and developing 
students’ basic skills. Several related practices underlie the success, while others help explain why the 
value-added scores, particularly in science and English Language Arts (ELA), remained lower than other 
high schools in the district in our study findings. 

A key driver of the school’s success was its strong leadership and the principal’s agenda to personalize 
relationships. This was complemented by the loyalty and trust of the staff, and the school-wide buy-in to 
the principal’ s personalization agenda. While the autonomy given to teachers engendered trust and buy-in 
to the principal’s goals, it also appeared to support an “anything that works” instructional environment 
lacking the supports that might increase instructional rigor. The principal acknowledged it was time to 
begin focusing on rigor, but used indirect strategies such as talking to individual teachers about 
observation ratings rather than through a concerted, stated effort so as not to over-stress teachers. 
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Riverview High School 

A public high school with a higher value-added ranking, Riverview is one of the larger and higher 
performing high schools in the district. Its special programs that appeal to many students include a large 
number of advanced courses, the Gold Seal Programs of Choice, and vocational training in areas such as 
agriculture and horticulture, construction, and information technology. The advanced courses attract 
primarily white, higher-income students, often from outside the school’s assignment zone, while regular- 
level courses have a greater proportion of minority and lower-income students. On the value-added 
rankings, Riverview was third overall among the district’s 13 high schools. However, it ranked sixth for 
free-lunch-eligible students and 12 th for Hispanics. 

Findings from our case study work offer insight into the school’ s success as well as the need for 
improvement. Across multiple interviews, students, faculty, and district leaders acknowledged the 
persistence of “two schools within the school,” a problem Riverview addressed with two broad strategies: 
to encourage more students to take honors and Advanced Placement (AP) courses and to protect and 
increase the quality and number of these courses. Faculty also described ongoing support for a wide 
variety of extracurricular activities as a key strategy for engaging students socially and academically and 
for building respect between students and teachers. While evidence exists that these two strategies have 
increased opportunities to learn for some students, they have failed to reach many disengaged students. 

Lakeside High School 

Described as a community school, Lakeside High School, a higher value-added school, has one of the 
highest concentrations of poverty in the district and serves a predominantly Hispanic population. 
Participants described the need for the school to “take care” of students’ emotional needs, noting that 
students sought stability at the school when it was lacking elsewhere in their lives. The value-added 
indicators ranked it near the top of the district in math and science, both for ah students and most 
subgroups. The value-added indicators placed it near the middle of the district for reading. Rankings for 
ELL students were low, however, particularly in science. 

Study participants indicated the school had improved significantly over the last several years, starting 
with the previous principal and continuing with the current leadership. A driving feature of the school was 
a shared and systemic focus on helping students to take responsibility for their own learning. Promoting 
students’ ownership and responsibility at Lakeside had both social and instructional aspects, including 
downplaying traditional modes of instruction in favor of a more student-centered framework and 
cooperative learning activities to engage students more actively. The school also stressed a culture of 
learning to hold students accountable and support them through systematic but personalized interventions. 
The administration fostered a culture of trust and positive climate that facilitated school processes and 
practices by listening to the concerns of faculty and staff and supporting their professional growth. 
School-wide practices included: the Lakeside Code, a set of expectations for students and teachers; 
Learning Time, a lunchtime tutorial system; Assignment Logs, a shared template for students to monitor 
their progress; and the Intervention Committee, which provided supports for students who were not 
meeting expectations. Tracing the development of recent improvement activities makes clear that the 
shared vision and teacher buy-in built over several years and the introduction of current structures 
formalized this vision of student ownership and responsibility. 



Main Findings. 


What differentiated the two HVA schools from the two LVA schools were practices that helped students 
take ownership and responsibility for their own academic success. Teachers and other adults in the HVA 
schools scaffolded students’ learning of both academic and social behaviors to guide them in assuming 
ownership and responsibility for their academic success. However, this strategy alone did not explain 
above-average gains for students. The schools also developed an integrated system of academic press (the 
encouragement of students to achieve) and support (resources to foster academic success) that was 
facilitated by a set of school-wide facilitating conditions. This involved promoting self-efficacy by 
changing students’ beliefs and attitudes and engaging them to do challenging academic work. Thus, self- 
efficacy and engagement are considered indicators of student ownership and responsibility, while 
academic support and press are strategies used to develop student ownership and responsibility. Figure 1 
presents a diagram of the theory of action behind the design challenge. This figure is intended to illustrate 
how our findings suggest the elements of developing student ownership and responsibility fit together. 
While our data do not permit causal claims, our findings are consistent with findings from other research. 
As illustrated in this figure, concerted school efforts to develop an environment of both academic press 
and support work to increase outcomes such as student self-efficacy and engagement. The intermediate 
outcomes of self-efficacy and engagement reinforce each other in a reciprocal relationship, and ultimately 
influence student achievement outcomes. These processes are supported by a set of school-wide 
facilitating conditions. 
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Figure 1 : Increasing Student Ownership and Responsibility for Academic Outcomes 



• Shared vision 

• Aligned and coherent structures 

• Trust 

• Faculty and student stability 

• Care and relationships between students and teachers 

• Teacher accountability 

• Individual and collective teacher efficacy 

• Safe and orderly environment 


Defining Student Ownership and Responsibility 

To help begin the work of the design challenge we provide definitions and specific examples of how our 
case study schools addressed student responsibility and ownership for academic learning. In this section, 
we describe what we mean by student ownership and responsibility and how schools can create 
conditions to develop it. We also make connections to the broader literature on these issues to show how 
our findings build off past work. The next section presents the evidence from this study that illustrates the 
importance of student ownership and responsibility. 

Increasing student ownership and responsibility for their academic success means creating a set of norms 
and school wide practices that nurture a culture of learning and engagement among students. Taking 
responsibility means becoming personally invested in one’s own learning and committed to 
understanding the work. Encouraging such a focus involves building students’ confidence and 
understanding of how they can take responsibility for their own academic success. We emphasize two 
activities important for this: 1) changing students’ beliefs and mindsets to increase self-efficacy (that is, 
an individual’s beliefs about his or her ability to perform behaviors that should lead to expected 
outcomes) 10 and 2) engaging students to do challenging academic work. 


Research shows that students who have strong, positive mindsets and a high degree of self-efficacy 
exhibit more positive academic behaviors, choose more difficult tasks, have higher engagement with 
academic work, demonstrate more persistence despite setbacks, and have higher achievement across 
academic areas . 11 Such students also demonstrate both behavioral and academic engagement. 1 " 

Behavioral engagement involves the basic behaviors expected in school, such as coming to class prepared 
and completing assigned tasks. They are important predictors of student achievement and, thus, predictors 
of whether students will graduate or drop out . 13 Academic engagement is defined as student investment in 
learning and the desire to challenge oneself. Students who are cognitively engaged exhibit strategic or 
self-regulating behaviors, are focused, and ask questions to clarify their understanding. Such students use 
strategies including rehearsing, summarizing, and elaborating in order to organize and understand the 
material . 14 

When students have a sense of ownership and responsibility for their learning, they: 

• Believe they can succeed at challenging academic tasks. 

• Are personally invested in academic success — both with the immediate learning task before them 
and in long-term outcomes such as college and career readiness. 

• Believe it is up to them and in their control to succeed in school. 

• Are able to identify and work toward learning goals with self-direction, productivity, and 
initiative. 

• Demonstrate their sense of responsibility through behaviors such as coming to class prepared, 
completing assignments well and on time, making up missed work in a timely manner, and 
seeking additional help when they are struggling (i.e., going to tutoring). 

• Demonstrate their investment through engagement in class, asking questions when they are 
confused, monitoring their own learning, and attempting to master material with which they 
struggle. 

• Demonstrate life skills such as initiative, self-direction, productivity, and accountability. 

Changing Mindsets to Increase Self-Efficacy 

Bandura 13 defines self-efficacy as people’s beliefs about their ability to perform behaviors that should 
lead to expected outcomes. Individuals with high academic self-efficacy are more likely than those with 
low self-efficacy to choose more difficult tasks, to expend greater effort, to exhibit more self-regulatory 
strategies, and to persist longer on these tasks . 16 Self-efficacy has also been shown to predict academic 
achievement across academic areas and levels . 17 

Our definition of student ownership and responsibility also involves conceptions of the student mindset, 
in particular the presence of an academic and flexible mindset. We draw on Dweck’s 18 research on the 
differences between people who ha ve fixed mindsets and those who have flexible mindsets. Students with 
a flexible mindset believe that intelligence can be developed through effort, while students with a fixed 
mindset believe that intelligence is static and indelible. Students with a flexible mindset respond 
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positively to challenges and persist in the face of adversity while those with a fixed mindset get defensive 
or give up easily. Similar to self-efficacy and a flexible mindset is the concept of an academic mindset 
that shapes how students see themselves in relation to intellectual work . 19 We use the term academic self- 
efficacy to capture these various related ideas because it most closely captures the sense that students 
believe they have the capacity to succeed academically and ties their behaviors to expected outcomes. 

Developing Student Engagement 

Student engagement is a multidimensional construct that includes cognitive and behavioral forms of 
engagement." Behavioral engagement represents students exhibiting the basic behaviors that are 
expected of them in school, such as coming to class prepared and doing the tasks set out for them. 

Students are cognitively engaged when they are also putting forth the mental effort to fully understand the 
work at hand. While both reflect dimensions of a student’s behavior, cognitive engagement behaviors are 
often more difficult for an outsider to observe. These two dimensions are sometimes referred to as 
substantive and procedural engagement . 21 While we discuss cognitive and behavioral engagement as two 
dimensions, they may also be considered a continuum of student engagement that ranges from the most 
teacher-directed to the most student-initiated where an engaged student might participate autonomously . 22 
The extreme end of cognitive engagement can be characterized as such total involvement in an activity 
that a person loses awareness of time and space . 23 

Cognitive engagement is defined as student investment in learning and the desire to exceed requirements 
and challenge oneself. Students who are cognitively engaged are committed to their learning and to 
understanding the work at hand. They exhibit strategic or self-regulating behaviors, are focused and ask 
questions to check or clarify their understanding. 

Cognitive or intellectual engagement relates to student investment in learning and a desire to exceed 
requirements and be challenged. Cognitive engagement is indicated when students exhibit strategic or 
self -regulating behaviors as they engage in academic tasks . 24 Cognitive engagement is also demonstrated 
by the ways in which students invest in the instructional tasks laid out by teachers and their concentration 
on those tasks . 25 Cognitively engaged students exhibit self-regulation or the ability to be strategic about 
how they study. Further, cognitive engagement includes problem solving, preference for hard work, and 
positive coping in the face of failure. Such students use metacognitive strategies to plan, monitor, and 
evaluate their cognition when accomplishing tasks and learning strategies. These include rehearsing, 
summarizing, and elaborating in order to memorize, organize and understand the material. 26 The deeper 
the strategies, the greater the cognitive engagement is. This manifests in greater mental effort, more 
connections between ideas, and greater understanding overall. 

We define behavioral engagement as positive conduct (e.g., following the rules, adhering to classroom 
norms, the absence of disruptive behaviors such as cutting class, skipping school, or getting into trouble); 
involvement in learning and academic tasks and behaviors including effort, persistence, concentration, 
attention, asking questions, and contributing to class discussion; and student participation in school- 
related activities (e.g., athletics or school governance ). 27 These forms of behavioral engagement are 
important predictors of student grades, which then predict graduation and dropping out. 2S 

Students who are behaviorally engaged do things that demonstrate their commitment to their learning. For 


example, they exert effort, are persistent, pay attention, participate in positive ways, follow school rules, 
are involved in learning and academic tasks, are on task, attend school, and get to class on time . 29 
Further, behaviorally engaged students complete their classwork and homework, organize their materials, 
comply with school rules, persist, participate and take initiative in classroom activities, and pay 
attention . 30 

School Strategies to Develop Student Ownership and Responsibility 

It is important to note that while student ownership and responsibility are measured by a set of outcomes 
at the student level, our research indicates that student ownership and responsibility resulted from 
concerted school efforts. Teachers and other adults in the school scaffolded learning of both academic and 
social behaviors that guided students in assuming ownership and responsibility of their academic success. 
Both of our higher value-added case study schools provided this scaffolding through integrated strategies 
of academic press and academic support. 

We define academic press as the degree to which both the school and the classroom environment push 
students to achieve. Academic press includes staff expectations, school policies, and practices, norms, and 
rewards generated by staff and students . 11 It exists when teachers expect students to work hard by (e.g., 
do more challenging work or attempt more challenging problems). Academic press also includes the push 
to get students into higher-level classes. When academic press is present it is part of the “nature of teacher 
norms toward student instruction at the school ”. 32 Academic press is related to gains in student 
achievement, particularly in low-SES schools. 33 

Academic support is the degree to which the school and classroom environment provide the resources 
students need to succeed academically — that is, to meet the demands created by academic press. Teachers 
can directly support students in developing ownership and responsibility by giving them personalized 
academic support . 34 Academic support can take many forms. It includes elements of curriculum (when 
schools provide challenging academic courses), the effective organization of time (such as extended 
learning opportunities), effective use of personnel to target individual student needs, rewards for student 
academic success, and the use of authentic and formative assessment . 35 Classroom instruction is critical. 

It provides academic support through collaborative, engaging activities that are relevant to students’ lives, 
a source of empowerment for students, designed around authentic questions, and focused on higher-order 
thinking skills . 36 Further, there is evidence that teachers can instruct students in strategies for engaging 
cognitively and behaviorally . 37 

In short, increasing student ownership and responsibility requires a commitment by teachers and the 
school as a whole to a scaffolded approach. Educators need to establish an environment of academic press 
and support to help students take ownership of their learning. We outline four attributes of schools that 
succeeded at increasing student responsibility and provide examples of strategies they used. 

• Teachers and other school personnel have high academic expectations for students. 

o School personnel hold students accountable to high academic standards by 

communicating clear and consistent expectations for performance and explaining the gap 
between those expectations and a student’s current standing. 
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o Teachers use instructional strategies and learning goals that push students into higher- 
level thinking. 

o Teachers create a sense of urgency among students to work productively during class 
time (i.e., giving students time cues to complete tasks). 

o The school day is structured to maximize and protect academic learning time. 

o School personnel encourage all students to take challenging courses and actively identify 
students who could succeed in more challenging courses. 

o Teachers maximize productive learning time (i.e., starting class on time, minimizing 
transition time during activities). 

Teachers and other school personnel provide instructional supports to help students meet high 
expectations. 

o Teachers use instructional strategies that require students to explain, analyze, problem 
solve, and produce something rather than applying formulaic procedures. 

o Teachers use authentic instructional strategies that emphasize the relevance to students’ 
current and future lives. 

Teachers and other school personnel provide organizational supports to help students meet high 
expectations. 

o The school day is organized to provide opportunities for struggling students to get extra 
help. 

o Teachers and other school personnel identify students who are struggling and develop a 
plan to intervene and provide additional supports. 

Teachers and other school personnel use techniques to deeply engage students in academic work. 

o Adults in the school model and explicitly teach students the behaviors that demonstrate 
investment and a sense of responsibility. 

o Teachers empower students by letting them lead classroom activity and discourse. 

o Teachers build on students’ intrinsic motivation by allowing them to apply the skills 
being taught in class to their areas of interest. 


o 


Teachers equip students with skills and strategies to learn how to learn. 



Evidence on Student Ownership and Responsibility 

This section presents the evidence from this study that illustrates the importance of student ownership and 
responsibility and the environment of academic press and support that our case study schools established. 
We draw on all four of our case study schools and emphasize the characteristics that appeared to 
differentiate the higher and lower value-added schools (HVA and LVA schools, respectively). In some 
cases, due to recent improvements in Valley through the turnaround efforts, we describe how Valley and 
the two HVA schools differ from Mountainside. Further, because one HVA school — Lakeside — had the 
most systematic and explicit focus on increasing student responsibility, we emphasize findings from 
Lakeside to underscore how one of the most economically disadvantaged schools in the district developed 
and sustained a coherent and integrated focus on helping students assume ownership of their learning. 

Our qualitative data suggest that both HVA schools had stronger and more systemic practices, policies, 
and resources to establish an academically rigorous school environment where students were pressed to 
achieve and supported in doing so. Indeed, one higher value-added school focused explicitly on 
increasing student ownership and responsibility for their learning. The vision shared by adults of student 
ownership and responsibility entails both changing the cultural/climate and instruction, including a focus 
on moving away from traditional modes of instruction to more meaningful, student-centered, and 
cooperative learning activities that require students to be actively engaged in their learning. This vision 
was led by the current principal, but had developed over several years. Several key personnel had realized 
that systems previously created in the school had led students to depend on teachers and other adults, 
training them to rely too heavily on others for their learning. School leaders decided to tackle that 
challenge in order to push for greater improvement in student achievement. The efforts to increase student 
ownership and responsibility focused on building a culture that holds students accountable for their 
learning and supports them through systematic but personalized interventions. Lakeside’s levers for 
academic press were the Lakeside Code, Learning Time, and the focus on student ownership and 
responsibility to try to enforce high expectations in all classes. (See the Promising Practices section below 
for a full description.) 

For example, the Lakeside Code, which outlines expectations for student conduct, focuses on academic 
and instructional behaviors rather than discipline or social behaviors. (See also the section below on the 
Lakeside Code.) Lakeside teachers, students, and administrators described academic behaviors as the 
heart of the student and teacher accountability mechanisms. School participants reported a strong 
perception that consequences existed if they failed to meet standards. Similarly, rewards were provided 
for meeting accountability standards. Notably, adults in the school described a relationship between 
student behavior and academic performance, recognizing that behavior reflects underlying academic 
issues. This belief undergirded academic-first responses to problems that were not ostensibly academic. 
Lakeside also provided systemic support structures to help students meet their academic expectations. 
While teachers across case study schools described being available to students for tutoring, Lakeside 
established an extended lunch period to encourage tutoring as the norm. Another key feature of 
Lakeside’s academic support system was the Intervention Committee, which worked with students who 
were not meeting standards to determine the root causes of their difficulties and develop a plan to address 
them. 
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The other higher value-added school, Riverview, also showed evidence of a strong student culture of 
learning, at least among the honors students who took the initiative to form study groups, tutor each other, 
and work collaboratively to master challenging material, often after school. Honors students also reported 
having been approached by lower-level peers to provide tutoring, outreach supported by adults in the 
school who allowed the use of classrooms for such engagement. Although this culture of learning was 
heavily influenced by parental press for high academic standards, even in Riverview where many students 
are college -bound, there was evidence of concerted strategies to increase student engagement to achieve 
school-wide rigor. Academics are described as the “driving goal” in Riverview, with concrete academic 
expectations. The school established academic press and support by highlighting its success with 
AP/honors courses to encourage more students to take those courses, with a concerted effort to keep the 
quality high. This outreach, which was targeted particularly at low-income and minority students, was 
described as a key lever to provide greater learning opportunities for a broad spectrum of the student 
population. One teacher illustrated this philosophy when she said the faculty was committed to taking 
students who are not “honors students” and making them into “honors students.” 

In contrast, the two LVA schools did not demonstrate a systemic focus on academic press and support. 
Participants in Valley High School reported they were working on these things, though not systemically. 
While Mountainside High School lacked student ownership and responsibility overall, small pockets 
existed in such programs as Junior Reserve Officer Training Corps (JROTC) and Advancement Via 
Individual Determination (AVID). One reported characteristic shared by the LVA schools was a “culture 
of multiple chances,” in which students could get several opportunities to make up for failure. While 
participants reported both positive and negative aspects to this practice, the limited student accountability 
it fostered supports the premise that academic press is a key difference between HVA and LVA schools. 
While all four schools provided credit recovery and other opportunities for students to make up failed 
assignments or courses, Lakeside and Riverview both were able to resolve the tension between supporting 
students and holding them accountable in ways that did not lower rigor. In contrast, LVA schools had 
only isolated examples of teachers pressing students and helping them take ownership of their academic 
success. 

Four items on our student survey capture aspects of student ownership and two focus on academic press. 
Of the items on student ownership, one focused on cognitive engagement and three on behavioral 
engagement. The academic engagement scale captures whether students get bored in class, find the work 
interesting, look forward to their classes, and work hard to do their best in class. On average, students 
were split between agreeing and disagreeing with the academic engagement items. The behavioral 
engagement measures are: study habits, responsibility-participation, and peer support for academic 
achievement. The study habits measure captures the extent to which students study and do homework. 
Students tended to agree that they were engaging in these behaviors. Respondents were most positive 
regarding the responsibility-participation items, which asked how many students in the school attend 
class, come prepared, and participate in class activities. The peer support measure captures whether 
students and their friends support each other academically by talking about what they did in class, 
preparing for tests together, helping each other with homework, and similar behaviors. For the academic 
press expectations scale, students were asked the extent to which they agreed with the following 
statements: my classes really make me think; my teachers expect me to do my best all the time; and my 



teachers expect everyone to work hard. In general, students agreed with these statements. The academic 
press challenges scale included items about the difficulty of class work, tests, and teacher questions and 
asked how often students felt challenged. Student surveys were administered district-wide to understand 
students’ perceptions. Across the district, students reported feeling challenged several times a month. 

In general, survey responses indicated stronger student responsibility and engagement at the HVA 
schools, though the evidence was not entirely consistent. Scale averages for Riverview were significantly 
higher than the district average, with the positive difference largest for study habits and participation and 
lower for engagement. This is consistent with our qualitative finding of a strong student culture of 
learning at Riverview. At Lakeside, the academic engagement and participation scale averages were 
significantly higher than district means, but the scales on study habits and peer support for academic 
achievement were lower. Results for the LVA schools were significantly lower in some areas and 
significantly higher in others. For instance, Mountainside had a statistically significant higher average for 
student study habits and peer support for academic achievement, compared with district means. 


Table 4. Student Survey Data on Academic Press and Student Ownership 


LVA HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Academic Engagement 

2.43***(-) 

2.52***(+) 

2.51**(+) 

2.49***(+) 

2.48 

(0.52) 

1-4 

Peer Support for 
Academic 
Achievement 

2.87***(+) 

2.72***(-) 

2.77**(-) 

2.89***(+) 

2.79 

(0.54) 

1-4 

Student Study Habits 

2.84***(+) 

2.67***(-) 

2.7i ***(-) 

2.85***(+) 

2.76 

(0.56) 

1-4 

Student Responsibility: 
Participation 

3.21***(-) 

3.51***(+) 

3.53***(+) 

3.60***(+) 

3.44 

(0.73) 

1-5 

Academic Press: 
Expectations 

3.04***(-) 

3.13 

3.07***(-) 

3.12 

3.11 

(0.53) 

1-4 

Academic Press: 
Challenge 

2.98 

2.95***(-) 

2.97**(-) 

3.01*(+) 

2.99 

(0.60) 

1-4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 


One survey result worth noting was that student perceptions of academic press were mixed at HVA 
schools. Riverview’ s scale averages exceeded the district mean for the category, but Lakeside had lower 
averages. While the differences are small, they were statistically significant in most cases. Valley also 
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showed some significantly lower scores on the academic press scales. 


The student survey also asks whether students participated in credit recovery, tutoring, and preparation for 
college entrance exams, and responses may shed additional light on student reports on academic press and 
support. For example, Mountainside students were most likely to report participating in credit recovery, 
suggesting less press “to do well the first time,” whereas the lower participation rates at the HVA schools 
suggest greater academic press. On the other hand, students in the HVA schools were more likely to 
participate in PSAT, SAT, and ACT preparation activities, suggesting more school-wide press to attend 
college. The effectiveness of Lakeside’s Learning Time tutoring program is evident in the high 
percentage of students who get tutoring. 


Table 5. Percent of Students Participating in Select School Programs 



LVA 


HVA 

District average 

Mountainside 

Valley 

Lakeside 

Riverview 

Tutoring 

58%*** 

43 %* 

75%*** 

26%*** 

44% 

PSAT, SAT, ACT Prep 

28%*** 

12%*** 

37%*** 

34%*** 

25% 

Credit Recovery 

12%*** 

8%** 

8% 

5%*** 

7% 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean value compared to the district mean. 


The CLASS-S observational data and student shadowing data also presented evidence on student 
engagement. Both the classroom observation data and the interview data suggested higher student 
engagement in the two HVA schools. The student shadowing data, however, showed a different pattern, 
with students in Valley being more actively engaged and students in the two HVA schools less actively 
engaged. Students in Lakeside were more often observed not engaged at all while students in Riverview 
were more often observed to be passively engaged. The differences between the shadowing data and other 
forms of data may be due to the sampling procedure for the student shadowing and the timing of the data 
collection. While the sampling procedure was similar across the four schools, with only 8-10 students per 
school we did not have a representative sample of students. Further, the classroom observation data were 
collected primarily in the fall and winter, when most of the interview and focus groups were conducted, 
while the student shadowing was done in the late spring. As this was near the end of the school year and 
state testing had already ended, we may not have found typical activities and levels of engagement. 



Table 6. Student Engagement Measures by School 



LVA Schools 

HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

Combined 

CLASS-S classroom observations: 
Student engagement 
Student shadowing data 

4.39** 

4.58 

4.83* 

4.88* 

4.67 

Actively engaged 

22.4% 

31.8%*** 

19.8%* 

18.3%* 

15.2% 

Passively engaged 

59.3 

54 . 4 *** 

56.3* 

63.7* 

54.8 

Not engaged 

18.3 

13.9*** 

24.0* 

18.0* 

15.2 


Note: The CLASS-S data came from observations of 603 20-minute segments of classroom observations. 
Classrooms in English/Language arts, mathematics, and science were observed. The observational rating is on a 
scale of 1-7. The shadowing data come from 1,360 5-minute segments of shadowing students throughout their core 
subject classes. 

* p<.05, **p<.01, ***p<.001. Tests for statistical significance were computed by comparing the school value to the 
value of the other three schools combined. 


Finally, we also examined administrative course-taking data as another indicator of academic press (see 
Table 7). We hypothesized that schools with a greater climate of academic press would have more 
students taking advanced courses and passing Advanced Placement (AP) exams. These data supported the 
finding that Riverview and, to a lesser extent, Lakeside, were more successful in getting students to take 
advanced courses and exams. Not surprisingly, given the fieldwork, Riverview had the highest percentage 
of students taking at least one AP course and passing an AP test. The other HVA school, Lakeside, had 
relatively low AP participation, although slightly more AP course -takers took the test than at Riverview. 
Both HVA schools had higher AP exam pass rates than the LVA schools. A recent increase in Valley, the 
percentage of students taking and passing an AP test buttressed findings from fieldwork about recent 
academic improvements. Lakeside has also recently increased the percent of AP testers passing the test. 

Enrollment patterns in honors and other advanced courses revealed a few differences among the schools, 
including mixed results in this area for the LVA schools. Valley experienced a drop in enrollment in 
advanced courses. Mountainside experienced a decrease in AP-related categories. Riverview showed an 
increase and had a total of 72% of students taking any advanced course (AP, honors, other advanced 
course), compared to around 51% in the other three schools. 
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Table 7. Course-Taking Patterns for Most Recent Three Years and Change Over Time 



LVA Schools 

HVA Schools 

District 

mean 


Mountainside 

Valley 

Lakeside 

Riverview 

Most recent 3 years 

% Taking any advanced class 

51% 

50% 

53% 

72% 

58% 

% Taking Honors 

47 

45 

48 

70 

51 

% Taking AP Class 

20 

21 

14 

34 

23 

% AP Takers who take the 

31 

38 

55 

53 

42 

exam 

% of AP Testers who pass 

11 

12 

25 

65 

28 

Two-year change 

% Taking any advanced class 

5% 

-8% 

10% 

-1% 

8% 

% Taking Honors 

4 

6 

13 

-1 

7 

% Taking AP Class 

-6 

5 

4 

1 

5 

% AP Takers who take the 

-2 

7 

3 

2 

5 

exam 

% of AP Testers who pass 

-6 

17 

16 

0 

6 


Note: These percentages represent the percent of all students in the school, although the availability of AP courses is 
not even across grades. The data on the most recent three years is an average of 2008-09, 2009-10, and 2010-11. The 
change over time data reflect changes from 2008-09 to 2010-11. 


School-wide Facilitating Conditions 

The Fort Worth ISD case study high schools suggest there are eight key conditions that sustain and 
integrate the school-wide strategies to increase student ownership and responsibility. They are consistent 
with the larger body of research on characteristics of effective schools: 

• A shared vision 

• Aligned and coherent structures 

• Trust 

• Care and positive relationships between students and teachers 

• Faculty and student stability 

• Individual and collective teacher efficacy 

• Teacher accountability, and 

• A safe and orderly environment 


Each of these school-wide facilitating conditions is described below, and supporting evidence provided 
from our case study schools. 





Shared Vision 

Effective schools are mission-driven organizations; they have a clear shared vision that animates daily life 
in the school. Leaders can articulate a vision for learning and hold high expectations for all students. 38 
School improvement efforts are enhanced when teachers and others in the school share the school-wide 
vision. 

The evidence from both case study HVA schools indicates that they had clear, shared goals that linked 
desired outcomes with strategies to achieve those outcomes. Notably, participants in both higher value- 
added schools identified a limited number of goals, while participants in the lower value-added schools 
reported multiple, sometimes inconsistent, goals. More important, goals in the higher value-added schools 
differed qualitatively. They not only wanted to raise student achievement, but also had a plan for how 
they would reach that goal. The vision in Lakeside focused on pressing students to take responsibility for 
their own learning and the vision in Riverview focused on promoting academic excellence through 
advanced course-taking accompanied by concrete academic expectations. 

Aligned and Coherent Structures 

The shared vision should not just be a set of ideals, however. It should include a coherent and consistent 
set of school-wide and classroom-level structures that are aligned with the vision. While classroom-level 
structures are necessary to ensure that new practices shape core instructional activities, school-wide 
structures will sustain and support teachers in implementing those activities. Lor example, instructional 
program coherence exists when a school 1) develops a common instructional framework with consistent 
expectations, materials, and strategies; 2) aligns teacher recruitment, evaluation, and professional 
development structures to the common instructional framework; and 3) strategically garners and allocates 
internal and external resources toward implementing this framework. 39 Prior research suggests schools 
that increased their instructional program coherence improved twice as fast as less coherent schools. 40 

Both higher value-added schools supported their school-wide vision with aligned and integrated school- 
wide practices. In Lakeside, several school-wide structures and practices, such as the Lakeside Code, 
Learning Time, and Intervention Committee, supported the shared vision of increasing student ownership 
and responsibility for learning. Lurther, these unique structures did not exist in isolation. Riverview High 
School also had structures aligned to its shared vision, creating a culture that reflected and advanced the 
goal of academic excellence and created considerable cohesion. Although Riverview had less strategic 
planning than Lakeside, there was a backdrop of action in the sense of getting students to excel. In 
contrast, Mountainside showed little evidence of convergence around any single effort or coordinated set 
of efforts to support school goals. Lurther, numerous participants indicated that the lack of 
communication or connection among adults in the school made it hard to meet the needs of students. 
Valley provided school-wide structures to support personalization and relationship-building, but 
structures to support the goal of increasing rigor were largely limited to one department. 

The teacher survey data on instructional program coherence supported the broader themes that have 
emerged from the interviews (see Table 8). Instructional program coherence includes items that capture 
the coordination and continuity of programs, curriculum, instruction, and learning materials within the 
school. Teachers from Lakeside, Riverview, and Valley high schools, for instance, rated the coherence of 
the school’s instructional program as higher than the district average. 
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Table 8. Teacher Survey Data on Instructional Program Coherence 


LVA 


HVA 





District 

Mountainside 

Valley 

Lakeside Riverview 

Mean (SD) 

Instructional Program Coherence 2.04***(-) 

2.80*(+) 

2.88*(+) 2.87***(+) 

2.63 (0.63) 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 
Note: The scale range is from 1-4. 


Trust 

Relational trust — which is the presence of mutual respect, personal regard, and a belief in the competence 
and integrity of others in the school — is a key contextual element of schools that can facilitate school 
operations and improvement activities. 41 When students trust their teachers and perceive them to be fair, 
they have higher academic motivation, improved behavior, and are less likely to drop out. 42 Trust 
between adults is also a resource for school improvement as it creates a safe environment for teachers, 
administrators, and other school personnel to take risks, try new ideas, and receive constructive 
criticism — that is, to learn. 4 ’ Although trust can facilitate school improvement efforts, the absence of 
significant trust does not mean improvements cannot be made; indeed school-wide improvement 
initiatives can also work to help develop trust among school personnel. 

Our data suggest that three of the four case study schools had sufficient amounts of trust to support school 
improvement efforts, particularly in Lakeside and Valley, one higher and one lower value-added school. 

In both schools, teachers spoke positively about each other and, in nearly all cases, about the 
administrative team. Moreover, faculty reported feeling respected by administrators, which in turn 
enhanced their respect of administrators. The culture of trust in both schools was fostered by two- 
directional communication between administrators and teachers, as well as collegiality and collaboration. 
In Riverview, an atmosphere of trust among adults was not described as explicitly as in Lakeside and 
Valley, although there was a general sense that students trusted their teachers and teachers trusted each 
other as well as school administrators. In contrast, trust between adults was particularly weak at 
Mountainside and deteriorated as the year progressed. The distrust reportedly stemmed from inconsistent 
disciplinary action by the administration, inconsistency in instructional initiatives (i.e., a sense that new 
initiatives were not given time to play out before another initiative was issued), lack of trust in 
administrators’ evaluations, as well as a sense among some that that the administration did not wholly 
value their work. 

The culture of trust evidenced at Lakeside provides the clearest example of how trust can facilitate the 
successful implementation of school-wide initiatives such as increasing student ownership and 
responsibility. Trust allows administrators and department heads to hold teachers to high expectations 
without significant resistance. Likewise, data are seen as a resource for professional responsibility and not 
information to punish teachers. Perhaps most important for understanding how practices such as the 
Lakeside Code work, the culture of trust increases teacher support for the shared vision and allows 


teachers to feel supported when holding students accountable and pressing them to be responsible for 
their learning. In contrast, while trust facilitated a sense of community in Valley, it was not sufficient to 
ensure high expectations for students or rigorous instruction. 

The teacher survey had three measures of trust: teacher-principal trust, teacher-teacher trust, and teacher- 
parent trust. The teacher-principal trust scale included items regarding principal visibility and 
accessibility and confidence in the school administration. The teacher-teacher trust scale included items 
about respect amongst teachers for providing high-quality instruction, respect for diverse professional 
opinions and practices, and ability to confide in teachers at the school. The teacher-parent trust scale 
asked teachers about the extent to which parents knew how to help their children with schoolwork at 
home and give timely responses to their requests. On average, teachers were most positive regarding 
teacher-teacher trust and least positive about teacher-parent trust. The survey data showed significantly 
negative average scores at Mountainside, which was consistent with the fieldwork finding of low trust in 
the school. The survey data from the other schools also echoed the interview findings of relatively high 
trust among teachers and between the teacher and principal. While trust of parents was not explicitly 
addressed in the interviews, it is not surprising that Riverview had the highest teacher-parent trust, given 
reports of higher parental involvement, which will be discussed in the Connections to External 
Communities section. 


Table 9. Teacher Survey Data on Trust 



LVA 

HVA 

District 
Mean (SD) 

Mountainside 

Valley 

Lakeside 

Riverview 

Teacher - Principal Trust 

2.3 1 ***(-) 

3.49***(+) 

3.46*(+) 

3.22 

3.14(0.79) 

Teacher - Teacher Trust 

2.92***(-) 

3.49*(+) 

3.56*(+) 

3.49*(+) 

3.37 (0.49) 

Teacher - Parent Trust 

1.84*(-) 

2.14 

1.94 

2.53***(+) 

2.13 (0.54) 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 
Note: The scale range is from 1-4. 


Care and Positive Relationships between Students and Teachers 

Strong, personalized relationships between students and teachers are a vital component of school 
organizational capacity. Students who have stronger relationships with their teachers are more likely to 
feel connected to the school, have greater academic motivation and achievement, and display more 
cooperative and less disruptive behavior, and less likely to feel alienated and drop out. 44 When students 
feel cared for, they are receptive to teachers and willing to engage in a reciprocal relationship. 45 Teachers 
can increase student motivation and learning by building rapport with their students, caring about them, 
and being enthusiastic. 46 

Our data suggest that while positive relationships between students and teachers are critically important to 
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student success, they cannot lead to high student achievement without a systemic focus on academic 
press. Valley illustrates this challenge. Relationships were strong, relatively widespread, and a specific 
focus of the school. However, without linking the relationships to academic learning (e.g., high 
expectations for students’ academic work, expectations for teachers’ rigor in instruction), the relationships 
often did not push students beyond attending school and passing their classes and state assessments. 

In contrast, the combination of a high floor for students’ academic expectation and positive teacher- 
student relationships allowed Lakeside to successfully implement student ownership and responsibility 
practices. A primary structure contributing to this success, Learning Time, encouraged teachers and 
students to build personal, non-academic relationships. Strong personal relationships also supported the 
success of the school Intervention Committee, where the goal was to understand the root cause of student 
failure and address problems in a student’s life that were inhibiting success. Riverview also had strong 
and relatively widespread positive relationships between students and adults, although they were stronger 
in the advanced/honors track. For example, as one student in a focus group described strong relationships 
with teachers: “the top students at our school are constantly in teachers’ rooms, figuring out assignments, 
going ahead, just trying to get it.” Positive relationships alone, however, did not relieve the persistence of 
“two schools within the school” at Riverview. At Mountainside, positive relationships were seen as 
important, but such relationships were present only in limited to pockets. 

Survey responses from participants at the schools indicated little variation in students’ experiences of 
personalization across the four schools. On the student survey, the personalization scale includes items 
that asked students how many adults in the school were willing to give extra help with their homework, 
cared about their academic progress, provided advice about graduation requirements, and helped with 
personal problems. On average, students reported that about 4-6 adults were willing to help in these ways. 
Riverview scored significantly lower than the district average on personalization, indicating that students 
had fewer adults they went to for help. In general, the similar averages across all four schools support the 
interview data findings that all the case study schools exhibited aspects of personalization even though the 
focus may have differed. 

One teacher survey measure, personalization-social, refers to the extent to which teachers know personal 
aspects of their students’ lives, such as their academic background and aspirations, their home life, and 
who their friends are. Across the district, teachers were most positive about this aspect of personalization. 
While the teacher survey data reflected no statistically significant differences in personal relationships 
between the case study schools, Lakeside had the highest averages for the structural support and extra 
help scales. This was similar to the interview data that noted a high level of academic support for students 
at Lakeside. 



Table 10. Student and Teacher Survey Data on Personal Relationships between Students and Teachers 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Student survey measure 
Personalization 

2.88 

2.83 

2.85 


2.85 

1-5 

Teacher survey measure 
Personalization - Social 

3.27 

3.18 

3.18 

3.25 

(1.02) 

3.25 

1-5 






(0.72) 



* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 


Faculty and Student Stability 

Similar to trust, school stability can exist at the student, teacher, and administrator level and serves as a 
key facilitating condition for school improvement activities. 47 Not only does mobility affect the students 
who are changing schools, but even non-mobile students have lower achievement in schools with high 
student mobility. 48 High levels of teacher and leadership turnover can also reduce a school’s 
organizational capacity by impeding the ability to create instructional coherence and eroding relational 
trust. 49 

Our data suggest that both leader and teacher turnover can negatively impact implementation of the 
components of effective schools. The particularly high turnover of teachers and leaders at Mountainside 
made it difficult for leaders to gain the trust of teachers and implement an effective reform agenda. In 
contrast, Valley had been relatively stable since implementing a turnaround model in which the new 
principal asked for a three-year commitment from those hired. As much of the academic and behavioral 
improvement in the school had been credited to the principal, stability in leadership had a clear impact. 
The two higher value-added schools had a stable core of leaders and teachers, although enrollment growth 
in Lakeside had led to an increase in new teachers over the past few years. Because of established 
routines of practice and shared goals in Riverview and Lakeside, their performance trajectories and 
improvement activities had been stable despite recent turnovers in leadership. 

Individual and Collective Teacher Efficacy 

Teachers’ sense of efficacy — which is a belief that teachers individually and/or collectively have the 
capability to achieve desired outcomes such as improved student achievement — serves as a form of 
school capacity and empowerment. 30 Teachers with a high individual sense of efficacy are more resilient 
in the face of challenging situations, have more focused instruction, and believe their actions can impact 
student learning. 51 On the other hand, teachers with low efficacy are more likely to deflect improvement 
efforts and blame poor performance on students’ lack of motivation. 52 Collective teacher efficacy exists 
when teachers believe that the faculty as a whole can impact student learning and achieve school-wide 
goals. 53 Collective teacher efficacy is related to student achievement, 54 possibly because teachers with 
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high collective efficacy are more likely to build positive relationships and respond positively to school 
improvement initiatives. 55 

Teachers at Lakeside appeared to have a greater sense of individual and collective efficacy. Whether this 
contributed to— or was an outcome of — the school’s success is not clear. This sense of efficacy did, 
however, support faculty and staff efforts to promote student ownership and responsibility with low- 
income Hispanic students. 

At Riverview, the other higher value-added school, teachers were most likely to report a sense of efficacy 
with students in honors and AP classes. While many in the school prided themselves on the inclusivity of 
their honors program, responses from some teachers suggested that they felt less efficacious with minority 
students not enrolled in honors courses and with their families. An administrator noted the need to narrow 
gaps between honors and regular classes by increasing student-centered activities/strategies and 
performance expectations in regular-level courses. Valley also appeared to place more blame on students 
and families for academic failure, although some teachers in Valley felt capable of supporting students’ 
social and emotional needs. Interviews with teachers in Mountainside suggested little sense of individual 
or collective efficacy. Multiple participants said their ability both to motivate students academically and 
to influence student achievement was heavily influenced by factors outside their control. 

Table 1 1 shows the efficacy data from the teacher survey. On average across the district, teachers agreed 
that they knew how to increase student retention of content covered in class, how to redirect disruptive 
students, and how to get through to even the most difficult or unmotivated students. While the efficacy 
mean was highest at Riverview, it was lowest at the other HVA, Lakeside. This finding appears to 
contradict the interview data finding that teachers at Lakeside had strong individual and collective 
efficacy. The lower score was driven by teachers at the school being more likely to note “agree” instead 
of “strongly agree” to survey questions, indicating still generally positive efficacy. It is also important to 
note that the differences were not statistically significant. 


Table 11. Teacher Survey Data on Teacher Efficacy 



LVA 


HVA 







District Mean 

Scale 


Mountainside 

Valley 

Lakeside Riverview 

(SD) 

Range 

Efficacy 

3.37 

3.39 

3.20 3.40 

3.34 (0.51) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 



Teacher Accountability 

To successfully implement strategies of academic press and support, schools need a systemic approach to 
teacher accountability. This may entail developing both individual and collective responsibility as well as 
internal mechanisms to ensure teachers are implementing desired practices consistently . 56 Schools and 
districts exercise internal accountability by establishing individual responsibilities , 57 which in turn enable 
the marshaling of resources to respond effectively to external accountability measures . 58 The consistent 
implementation of local expectations and responsibilities requires that leaders hold teachers accountable 
for implementing practices that align with the shared school-wide vision. This often requires frequent 
instructional observations and curricular discussions or other mechanisms . 59 

The two higher value-added schools had more systemic approaches to teacher accountability and 
integrated accountability for students and teachers. For example, teachers were held accountable for also 
holding students accountable and helping them to succeed. The increased teacher accountability was 
accompanied by pervasive administrative support. In Riverview, we found evidence across the faculty of 
a willingness to be accountable for student outcomes to one another or to their broader fields (e.g. AP 
subject areas), coupled with an openness to investigate new strategies or options. In Lakeside, the role of 
classroom walk-throughs illustrates the integration of teacher support and accountability. There, teacher 
observations were part of an ongoing, bilateral dialogue between teachers and leadership, a hallmark 
feature of the practice. Feedback from observations was looped, with observers asking guiding questions 
and expecting a response from teachers. In contrast, observations in the lower value-added schools were 
seen as merely proforma or perfunctory and done for accountability purposes rather than being designed 
to help teachers grow. Particularly in Mountainside, but to some extent in Valley, there was a sense that 
administrators used observations to catch teachers making mistakes. Teachers at Valley were largely left 
alone unless there was a problem. 

Data from the teacher surveys indicated that scales related to teacher accountability and principal 
effectiveness in supporting systematic performance accountability were particularly high in Lakeside and 
particularly low in Mountainside, supporting many of our assertions drawn from the qualitative data. The 
teacher accountability construct measures the extent to which teachers feel the following are true: there 
are consequences for teachers who don’t perform well, they get valuable feedback on their instruction, 
other teachers hold them accountable for their performance, and teachers who don’t do well are given 
opportunities to improve. The principal’s effectiveness at supporting systemic performance accountability 
captures the teacher’s evaluation of the principal in holding faculty accountable, advocating for holding 
students accountable for achieving at high levels, and challenging faculty that do not hold students 
accountable. Both HVA schools scored higher than their LVA counterparts on teacher accountability. 
Mountainside had the lowest mean scores across both systemic accountability-related scales, supporting 
many of our assertions drawn from the qualitative data. Scores for principal support for systematic 
performance accountability were mixed, for example, with Valley scoring higher than the district average 
and Riverview similar to the district average. The high regard teachers in Valley had for the principal may 
contribute to this high rating. Somewhat surprising, given the prior findings about observation and 
feedback, none of the case study schools were statistically different from the district average in teacher 
perceptions of instructional observations and feedback. 
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Table 12. Teacher Survey Data on Teacher Accountability 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Teacher Accountability 

2.45***(-) 

3.08 

3.22*(+) 

3.16**(+) 

2.98 

(0.60) 

1-4 

Principal Effectiveness at 
Supporting Systemic 
Performance 
Accountability 

2 20***(-) 

3.45***(+) 

3.53***(+) 

3.21 

3.07 

(0.82) 

1-4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 


Safe and Orderly Environment 

Schools must have a calm, safe, and orderly environment for the strategies and school-wide facilitating 
conditions to take root. Indeed, trust, caring relationships, and teacher responsibility require safe and 
orderly climates where student discipline is handled in a fair and consistent manner. 60 Schools 
characterized by safety and order see greater improvements in the essential components of effective 
schools that contribute to student outcomes. 61 Likewise, a consistent approach to behavior management is 
a building block for personalization of academic and social learning. 62 

A safe and supportive disciplinary climate has the potential to facilitate other activities, such as fostering 
trust. For example, teachers and students trust administrators when they see rules enforced consistently. 
Lakeside and Riverview demonstrated consistent behavior enforcement, although in Lakeside, 
discipline/behavior management was integrated into the focus on student ownership and responsibility. 
For example, the Lakeside Code, which serves as a student code of conduct, focused more on academic 
and instructional behaviors than discipline or social behaviors. While there was support for addressing 
small infractions, the strategies often included an academic component, such as completing homework or 
bringing a book to class. Student behavioral discipline was actively linked to academic discipline; school 
personnel sought to understand the “root cause” of student academic failure or misbehavior. 

As with positive relationships, consistent disciplinary enforcement is necessary, but it cannot promote 
student ownership and responsibility by itself. For example, at Valley Fhgh School, teachers gratefully 
reported that administrators consistently and fairly resolved discipline problems (facilitating relational 
trust), and Valley had significantly more disciplinary referrals than any other case study school. The fact 
that the referral rate had also increased in recent years, reflects the attention given to this by school 
leaders. 

Mountainside lacked consistent disciplinary enforcement and it led to lack of trust between the 
administration and teachers, as well as between the administration and students. This, in turn, inhibited 
other efforts to improve achievement. 


Several items on the student survey capture aspects of a safe and orderly environment. There are 
differences by schools, but no clear pattern of differences emerged between the HVA and LVA schools as 
a group (see table 13). Not surprising given the interview data, students in Riverview reported feeling 
safer and less bullying. Students in Valley and Lakeside reported more bullying. This is different than the 
teacher data where teachers at Lakeside reported less bullying than the district average. Students at 
Mountainside reported feeling less safe which is consistent with the interview data. 


Table 13. Survey Data on Safe and Orderly Environment 



LVA 

HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Student survey measures 

School Safety 

3.11 *(-) 

3.20 

3.19 

3.22***(+) 

3.18 

(0.56) 

1-4 

Bullying 

2.40***(-) 

2.65***(+) 

2.66***(+) 

2.48***(-) 

2.56 

(0.79) 

1-4 

Teacher survey measure 

Bullying/School safety 

2.68 

2.23 

3.20*(-) 

3.40 

2.36 

(0.88) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparisons 
tests between each case study school’s mean scale rating compared to the mean from the district’s other 12 schools. 


The teacher and parent survey also provided data on the safe and orderly environment. In terms of teacher 
perceptions of the student climate, the only measure available is about bullying/school safety, where 
teachers in Lakeside reported significantly less bullying compared to other teachers in the district. The 
bullying items asked parents to indicate whether their child had been bullied at the school or via texting, 
Internet, or phone. Note that this is not consistent with student reports of bullying as shown above. On 
average, parent responses across all four schools indicated that their students had been bullied, though 
none were significantly different from other parents in the district. 

There are several indicators in the administrative data that help us understand the school climate for 
students (Table 14). The four case study schools had similar student attendance rates, although the rates 
had risen in recent years at the HVA schools so both were slightly higher than at the LVA schools. The 
HVA schools also had fewer students with disciplinary action, suggesting fewer behavioral concerns. 
Riverview had seen a marked decrease in disciplinary referrals. 
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Table 14. Attendance and Disciplinary Behavior for Most Recent Three Years and Change Over Recent 
Two Years 



LVA Schools 

HVA Schools 

District 


Mountainside 

Valley 

Lakeside 

Riverview 

mean 

Most recent 3 years 

Attendance Rate 

90.2% 

92.1% 

94.1% 

93.1% 

91.9% 

S. w / "Disciplinary Action" 

30.0 

22.6 

18.5 

12.4 

17.8 

Two-year change 

Attendance Rate 

0.5% 

-2.2% 

0.2% 

-0.4% 

-0.8% 

S. w / "Disciplinary Action" 

-23.5 

-3.7 

4.1 

-10.1 

-8.6 


Finally, the CLASS-S observations include a measure of behavior management. These data also support 
the finding that the HVA schools had teachers with higher behavior management ratings than teachers in 
LVA schools. Further, Mountainside had the lowest behavior management scores, consistent with the 
interview data. 


Table 15. Mean CLASS-S Behavior Management Score by School 



LVA Schools 

HVA Schools 


Mountainside Valley 

Lakeside Riverview Combined 

Behavior Management 

5.05*** 5.25* 

5.91*** 5.61 5.44 

* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean comparison 
tests between each case study school’s mean rating compared to the mean from the district’s other 12 schools. 



Essential Components: Comparisons Between Higher and 
Lower Value-Added Schools. 

The above sections provide the evidence on the main findings from this study. Yet as we described in 
the introduction, we did not begin this study looking for evidence of student ownership and 
responsibility. Rather, our work was guided by what we call the essential components of effective 
schools and the findings described above emerged from this work. In this section, we describe the 
analytic framework that guided our work and present the evidence we examined on each essential 
component. 

We submit that far-reaching school improvement in high schools is rooted in a set of eight essential 
components that emerge from the literature on effective schools in general and effective high schools 
in particular. Schools succeed not because they adopt piecemeal practices that address each of these 
components, but rather because they organize their collective practices into a coherent and cohesive 
framework of aligned practices. In effective schools, these components are woven into the school’s 
organizational fabric to create internally consistent and mutually reinforcing reforms; their success is 
explained by more than the simple sum of their parts. The conceptualization of the Center's 
framework suggests that these essential components can work together in effective high schools to 
create deep connections, engagement, and attachment for both adults (leaders, teachers, staff) and 
students, to the work, norms, and outcomes of high schools. Similarly, the inability to effectively 
implement these components to high quality and high frequency can explain alienation, 
disengagement, and lack of effort in high schools for students and adults . 63 The eight components are 
organized into two broad categories. First are the “anchors.” These two components hold together the 
other six and cut across them. The first is Learning-centered Leadership, which entails the extent to 
which leaders hold a vision in the school for learning and high expectations for all students and focus 
all leadership. 64 , The second is Rigorous and Aligned Curriculum, which focuses on the content that 
secondary schools provide in core academic subjects, including both the topics students cover as well 
as the cognitive skills they must demonstrate in each course. 65 . While none of the components in and 
of itself is sufficient for an effective high school, learning-centered leadership and rigorous and 
aligned curriculum are the aspects upon which the other components can be built and, in an effective 
school, hold strong influence over how the other components are enacted. The remaining essential 
components constitute the necessary elements to develop engagement, commitment, and shared 
norms and values. The third component is Quality Instruction, the teaching strategies and assignments 
that teachers use to implement the curriculum and help students to reach high academic standards. 66 . 
The fourth is Personalized Learning Connections, the development of strong connections between 
students and adults that allow teachers to provide more individual attention and talk with each student 
regarding unique circumstances and learning needs 67 as well as developing students’ sense of 
belonging . 68 The fifth essential component is a Culture of Learning and Professional Behavior. This 
refers to the extent to which teachers take responsibility for events in the school and their students’ 
performance, and the degree to which they collaborate in their efforts through such activities as 
school-wide professional development . 69 The sixth component is Connections to External 
Communities, the ways in which effective secondary schools establish meaningful links to parents 
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and community organizations and relationships with local social services, as well as student work 
experiences in the community . 70 Another component is Systemic Use of Data, including the use of 
data to inform classroom decisions and multiple indicators of student learning . 71 The final essential 
component is Systemic Performance Accountability, which includes external and internal structures 
that hold schools responsible for improved student learning. External accountability refers to the 
expectations and benchmarks from state and national bodies, while internal accountability consists of 
district- and school-level goals . 72 This section presents findings from our study focused around these 
essential components. We first summarize the prior literature on each essential component and then 
describe key differences between HVA and LVA schools with respect to that component, using 
interview, survey, administrative, and observational data. 

Learning -centered Leadership 

Learning-centered leadership focuses on cultivating, supporting, and improving the essential 
components of effective schools. Prior studies suggest links to increases in students’ learning when 
leaders organize their schools by articulating an explicit school vision, generating high expectations 
and goals for all students, and monitoring their schools’ performance through regular use of data and 
frequent classroom observations . 73 Principals’ effects on student learning are also likely mediated by 
their efforts to improve teacher motivation and working conditions 74 as well as the efforts to hire 
high-quality personnel . 75 Finally, research suggests that principals can play important roles in 
implementing instructional reforms. Quinn 76 found that teachers more frequently engaged in new 
instructional strategies when their principals actively worked to secure curricular materials and acted 
as instructional resources for instructional reforms. 

Following a distributed perspective of leadership , 77 effective learning-centered leadership transcends 
one person or any specific role, and is visible across multiple actors and structures in schools. (Our 
definition of leaders in these schools includes administrators, lead content teachers, department 
chairs, and leaders of other groups such as professional learning communities.) Effective learning- 
centered leadership is “coordinated and patterned ," 78 aligned and consistent with the other systems 
and components within the school. 

The case study data revealed that effective learning-centered leadership in the Fort Worth ISD context 
is characterized by a number of practices and approaches: 

Shared Understanding and Commitment to Goals 

Both higher value-added schools, Fakeside and Riverview, had clear goals and the interview data 
suggest that the leadership drives the practices, policies, and structures. Indeed, teachers and staff had 
a shared understanding of the principal’s goals. There was cohesiveness around the goals and, equally 
important, school-wide, systematic approaches for how to achieve them. In Fakeside, for instance, 
there was a clear link between the principal’s goal of students taking responsibility for learning and 
the structures (e.g., the Fakeside Code) that supported that goal. In contrast. Mountainside’s lack of 
cohesion around a sense of shared purpose and goals was evidenced by fragmented structures and 
communication and scant evidence of systematic efforts to accomplish these stated objectives. 
Teachers in Mountainside complained about lack of focus and vision and the “Christmas tree 


approach,” with a large number of unrelated programs and policies. In Valley, participants described 
many different school-wide goals, with less agreement among participants than was evidenced in the 
HVA schools. 

Openness to Dialogue and Teacher Involvement in Decision-Making 

Teachers and administrators in both Lakeside and Riverview described their schools as having a 
“family” culture. Leadership in Lakeside and Riverview (administrators, department heads) indicated 
that teachers were part of decision-making processes, and that teachers had input in decisions, 
although Riverview’s larger size made it seem more hierarchical while Lakeside had more direct 
engagement with teacher input. This type of open dialogue reinforces a culture of trust and mutual 
respect. Valley also had strong connections between principals and teachers and buy-in to the 
principal's vision, a leadership approach that acknowledges the importance of a shared culture. In 
contrast, Mountainside’s leadership did not foster open dialogue, trust, and teacher respect; the 
principal of Mountainside was referred to as a micromanager. 

Principals and Other Leaders Observe Classrooms to Focus on Quality Instruction. 

Instructional leaders spend time in classrooms and explicitly talk about expectations for classroom 
instruction, and provide feedback to teachers. The most salient focus of the leadership in the higher 
value-added schools is around quality instruction. Leaders spend time in classrooms, not just to 
evaluate, but as a mechanism to focus the school on the vision and goals of improving instruction. In 
Lakeside, both teachers and administrators commented about the high frequency of classroom 
walkthroughs and the helpfulness of resulting feedback. In Riverview, administrators were more 
likely to talk about classroom observations than the teachers, but both mentioned visits by the 
principal with emphasis on particular types of instruction. Multiple teachers, for example, indicated 
that the principal had recently talked with teachers about several topics, including the unsuitability of 
worksheets for instruction and the expectation for teachers to be present at the front of the class for 
about 10 minutes before diversifying instruction. The instructional focus and feedback role of 
classroom observations was not evident in Mountainside, where some teachers questioned the validity 
and usefulness of feedback. Valley teachers also felt they did not receive adequate feedback about 
their instruction. 

Distributed Leadership 

The administrative structures in the higher value-added schools, as well as at Valley High School, 
supported a coherent and shared focus on expectations for improving instruction. Through common 
planning time, departmental meetings, and the roles of lead content teachers (LCTs) and assistant 
principals, leadership teams could reinforce improvement goals. The LCTs, for example, arose from a 
district initiative to provide content-based expertise in their area and act as liaisons with the district 
content specialists who serve as resources for the core tested subjects. LCTs are classroom teachers in 
core content areas who are given additional release time to provide support to teachers in their content 
areas. In addition, the roles and responsibilities were carefully articulated to support the vision and 
goals of the school. Structures supported academic and social learning goals. In Valley for example, 
certain structures supported the focus on developing professional collaboration, including 9 th - and 
10 th -grade teachers that were teamed by student groups (e.g., honors, regular, special education, 
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English language learners). Mountainside, however, had a breakdown of distributed leadership as 
evidenced by inconsistent procedures and processes, a lack of order in the school, and a lack of clarity 
in roles and responsibilities, such as inconsistent disciplinary actions. 

Quantitative Data 

The teacher survey data on leadership supports the broader themes that have emerged from the 
interviews (see Table 16). Teachers from Lakeside, Riverview, and Valley high schools, for instance, 
rated the coherence of the school’s instructional program and principal effectiveness at supporting 
quality instruction as higher than the district average. Instructional program coherence includes items 
that capture the coordination and continuity of programs, curriculum, instruction, and learning 
materials within the school. The principal’s effectiveness at supporting quality instruction scale is an 
evaluation of their principal’ s effectiveness at maximizing instructional time, discussing instructional 
strategies, and monitoring instructional quality. For Valley, the mean was significantly higher than 
the mean for the rest of the district. This finding is consistent with the open dialogue and distributed 
leadership witnessed at Valley High School. The teacher survey results support the findings from 
Mountainside High School, where the scale means were significantly lower than the district means 
across both leadership indicators. 


Table 16. Teacher Survey Data on Leadership 



LVA 

HVA 

District 



Mountainside Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Instructional Program 
Coherence 

2_04***(-) 2.80*(+) 

2 . 88 *(+) 

2.87***(+) 

2.63 

(0.63) 

1-4 

Principal Effectiveness at 
Supporting Quality 
Instruction 

2.33***(-) 3 39 ***(_)_) 

3.21 

3.11 

3.00 

( 0 . 68 ) 

1-4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Effective learning-centered leadership is distributed leadership, with leaders consistently focused on 
the mission and goals. The culture of the school encompasses both high expectations and supports to 
reach those expectations. The supports are primarily in the form of feedback from classroom 
walkthroughs and distributed leadership structures where instructional improvements can be 
addressed, a culture where teachers are respected and listened to, and structures and processes that are 
aligned with the goals and consistently applied. Principals are visible drivers of these elements of 
effective leadership. 


Rigorous and Aligned Curriculum 

Rigorous and aligned curriculum focuses on the content that schools provide in core academic 
subjects 79 and is a second anchor that cuts across the other components. A rigorous curriculum is 
intellectually challenging, covers broad and deep content, and prepares students for college and 
careers. Curricular alignment addresses the degree to which curriculum standards, curriculum 
frameworks, and assessments work together at district, school, and classroom levels . 80 Further, it 
implies a strong link among objectives, assessments, and instructional activities and materials . 81 On 
the whole, high school curricula are driven by state standards, as required under No Child Left 
Behind . 82 Research on curriculum at the high school level centers around differences between 
vocational/technical curriculum or remedial courses and college preparatory curriculum, case studies 
of implementing new packaged curricula, the effects of increasing curricular requirements for 
graduation, and access to curriculum, specifically advanced courses, for different groups of students. 

There do not appear to be significant differences between HVA and LVA schools in terms of 
curriculum, or alignment, as these elements were defined by the state, provided by the district, and 
implemented similarly across the schools. While some faculty at one of the schools reported that they 
wrote some of the curriculum used district wide, the curricular materials made available to all four 
schools by the district were identical. 

Alignment 

All four schools used the Fort Worth ISD Curricular Frameworks as their primary source of 
curriculum and teachers tried to stay “on the same page” with each other and/or with the pacing 
guides provided by the district. Across the four schools, teachers did report choosing different 
activities than what was recommended in the Curricular Frameworks, but still adhered as much as 
possible to the content and learning objectives. When alignment is interpreted as whether the 
curriculum (lessons, activities, objectives) is aligned to the state standards (Texas Essential 
Knowledge and Skills, or TEKS) and assessments (STAAR and TAKS), the data suggest no 
differences among schools. Very little of the qualitative data speaks to whether the activities and 
lessons teachers used during class — either within or beyond the Curricular Frameworks — were 
aligned with the TEKS standards. Teachers described following the district Curricular Framework, 
with the implicit assumption that this meant they were aligned with the TEKS. The most significant 
form of misalignment with the Curricular Frameworks were statements heard across the schools, but 
particularly in Lakeside, Mountainside, and Valley, about the quick pace of the Frameworks. When 
alignment is interpreted as the degree to which teachers are aligned with each other, all four schools 
showed similar patterns of expectation and flexibility, in which teachers needed to be generally in line 
with the pacing of the Curricular Framework and with each other. No school’s teachers were free to 
completely disregard the district’s Curricular Frameworks, nor did any school require every teacher in 
a given subject to teach the same lesson on the same day or face consequences for being off -pace. In 
all four schools, common planning time and regular Curriculum Based Assessments (CBAs) 
facilitated alignment between teachers. 
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Rigor 

In the qualitative data, it appears that the HVA schools offered more Advanced Placement/honors 
courses and made more concerted efforts to encourage students to take advanced courses than the 
LVA schools. It is hard to distinguish whether this was the result —or a cause — of their HVA nature. 
Similarly, Mountainside, which offered the fewest advanced courses, was also significantly smaller, 
making it difficult to determine whether rigor or staffing issues were at the heart of this difference. 
While personnel at all schools discussed the need for rigor, there were differences in the instructional 
rigor and academic press between higher and lower value-added schools. Both HVA schools had 
levers to push for high expectations and rigor throughout the school. Riverview used its success with 
AP/honors courses to push more students into those courses while also keeping them rigorous. 
Lakeside used the Lakeside Code, Learning Time, and the focus on student responsibility to try to 
enforce high expectations in all classes. In contrast, Valley and Mountainside did not exhibit high 
expectations and had a culture of multiple chances that lowered expectations. Academic press was 
lower or non-existent at LVA schools compared to HVA ones. Lakeside and Riverview both found 
ways to resolve the tension between supporting students and holding them accountable in ways that 
did not lower rigor. The Lakeside Code and related structures did this in Lakeside. In Riverview, the 
press largely came from parents. 

Quantitative Data 

The district provided student-level data on course enrollment, allowing us to consider whether 
students are enrolling in more advanced courses than previously as a measure of rigor. We consider 
three types of advanced courses: Advanced Placement (AP) courses, honors courses, and any other 
course with “advanced” in its name. The AP data noted whether students took the AP test and 
whether they passed. 

We first consider AP course enrollment, test-taking, and pass rates. There are several stages of 
selectivity in AP data, represented in Figure 2. Out of a school’s entire population, only some students 
take an AP course, and only a subset of those students take the AP exam. Likewise, only some exam- 
takers pass. Thus, care must be taken when comparing AP pass rates across schools to consider the 
selection of the students who take the exam. 


Figure 2. Stages of AP Selectivity 
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As is evident in Table 17, the HVA schools were more successful in getting students through each of 
these stages of AP selectivity. Not surprisingly, given the fieldwork, Riverview had the highest 
percentage of students taking an AP course and passing an AP test. Lakeside had a relatively low 
percentage of students taking an AP class, although those students were slightly more likely to take 
the test than at Riverview. Both HVA schools had higher percentages of students passing AP tests 
than the LVA schools. At Valley, an increase in AP test-taking and passing was consistent with 
findings of improvement in the fieldwork data. Lakeside also had recently increased the percent of 
AP testers passing the test. 

Looking at patterns of student enrollment in honors and other advanced courses revealed few 
differences, although the LVA schools appeared to have recently decreased in this area. Riverview 
had by far the largest participation rates in any kind of advanced class. Lakeside’s three-year average 
was similar to that of the LVA schools for participation in any advanced or honors class but much 
higher for measures related to Advanced Placement. Riverview has a total of 72% of students taking 
any advanced course (AP, honors, other advanced course), compared to about 51% in the other three 
schools, and although Lakeside’s honors enrollments had increased substantially in the last two years. 


Table 17. Course-Taking Patterns for Most Recent Three Years and Change over Time 



LVA Schools 

HVA Schools 

District 


Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

Most recent 3 years 

% Taking any advanced class 

51% 

50% 

53% 

72% 

58% 

% Taking Honors 

47 

45 

48 

70 

51 

% Taking AP Class 

20 

21 

14 

34 

23 

% AP students who take the 

31 

38 

55 

53 

42 

exam 

% of AP testers who pass 

11 

12 

25 

65 

28 

Two-year change 

% Taking any advanced class 

5% 

-8% 

10% 

-1% 

8% 

% Taking Honors 

4 

6 

13 

-1 

7 

% Taking AP Class 

-6 

5 

4 

1 

5 

% AP students who take the 

-2 

7 

3 

2 

5 

exam 

% of AP testers who pass 

-6 

17 

16 

0 

6 


Note: These percentages represent the percent of all students in the school, although the availability of AP 
courses is not even across grades. The data on the most recent three years is an average of 2008-09, 2009-10, 
and 2010-11. The change over time data reflect changes from 2008-09 to 2010-11. 


The teacher survey data provide additional evidence of these differences in rigor. It is important to 
note that, in some instances, survey data overlap or complement qualitative data, while in other 
instances survey data supplement the interview data by addressing different pieces of the components. 
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The teacher survey data supplement the interview data because questions of postsecondary 
expectations and college preparation were not fully addressed in the interviews. Questions about 
postsecondary expectations capture whether teachers expect their students to go to college or attend 
postsecondary training and whether they think students themselves are planning to do so. Teachers in 
the LVA schools had significantly lower postsecondary expectations for their students than the 
district average. Teachers at Riverview had significantly higher expectations compared to other 
teachers in the district, agreeing, on average, that they expected their students to go to college or 
attend postsecondary training. 


Table 18. Teacher Survey Data on Rigorous and Aligned Curriculum Scales 



LVA 

HVA 

District 





Mean 

Scale 


Mountainside Valley 

Lakeside Riverview 

(SD) 

Range 

Postsecondary 

Expectations 

2.34*(-) 2.42**(-) 

2.56 3.04***(+) 

2.69 

(0.78) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


In the student survey data, the closest proxy for rigor is school-wide future orientation. This measure 
includes whether all students are encouraged to go to college and think about their future and whether 
teachers pay attention to all students, not just the top performers. There was some variation by school, 
though the differences in the mean scores were slight. The significantly lower average at Riverview 
indicated the general feeling that lower track students were not pushed and held to the same standard 
as the higher track students. The significantly higher average at Valley indicated that students were 
somewhat more positive about teachers helping all students to learn, plan for college, and stay in 
school. 



Table 19. Student Survey Data on Rigorous and Aligned Curriculum 


LVA 

HVA 

District 




Mean 

Scale 

Mountainside Valley 

Lakeside Riverview 

(SD) 

Range 

School-Wide Future Orientation 2.89 2.94***(+) 

2.89 2.88***(-) 

2.90 

(0.53) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Quality Instruction 

Quality Instruction encompasses the instructional strategies that teachers use to achieve high 
standards for all students. Much of the research discussing the quality of instruction at the high school 
level is descriptive, either explaining programs that have been developed and implemented to increase 
student achievement, particularly in math, or case studies describing the practices of effective 
teachers. Trends in this research cluster around common practices and specific classroom foci. 
Common practices include collaborative group work and inquiry-based learning (Staples, 2007), 
formative assessment (Brown, 2008), scaffolding, and introducing new concepts concretely through 
direct instruction in which each concept is broken down into small parts, taught, and then assessed for 
mastery before the teacher moves on (Alper, Fendel, Fraser & Resek, 1997). Foci include creating 
structures and classroom climate that allow students to try and fail without negative consequences 
(Alper, et al., 1997), making content not only relevant for real life, but important, and setting high 
expectations for all students (Boaler, 2008). 

The vast majority of more recent work on the quality of instruction has focused on developing 
frameworks and corresponding classroom observation rubrics. The framework guiding the Center's 
work on conceptualizing the quality of instruction in high schools is the CLASS-S. The CLASS-S 
articulates domains and dimensions of quality instruction, in which dimensions describe various 
aspects of each domain. The three core domains of the CLASS-S are Instructional Support, 
Emotional Support, and Classroom Organization, with a fourth domain, Student Engagement, as an 
outcome. Instructional Support includes teachers’ demonstration of their content understanding, the 
methods they use to facilitate student use of higher-order thinking skills, the quality of feedback 
teachers provide, and their use of instructional dialogue to facilitate content understanding. Emotional 
Support largely overlaps with the academic engagement aspect of Personalized Learning Connections 
and includes measures of positive and negative classroom climate, the teacher’s sensitivity and 
responsiveness to student needs, and the teacher’s regard for adolescent perspectives, i.e., the degree 
to which teachers provide opportunities for autonomy and leadership as well as relevant applications 
of content. Finally, Classroom Organization includes behavior management, productivity or the 
maximization of learning time, and teachers’ use of a variety of instructional learning formats to 
maximize student engagement. 
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This framework, as well as others, suggest that high-quality instruction is rooted in a notion of 
engaged learning (instructional dialogue, feedback, responsiveness), whereas low-quality instruction 
consistently allows students to be passive, and disengaged as learners (seatwork, receivers of 
information, and limited accountability for learning). 

Interview Data 

The interview data provide insight into what the school stakeholders consider quality instruction. 
Across the four schools, student engagement was commonly considered important for high-quality 
instruction. However, in the higher value-added schools, especially Lakeside, engagement was linked 
to student responsibility and active participation in lessons, whereas in the lower value-added schools, 
it was more often linked to specific strategies such as real world connections or bell-to-bell 
instruction. The first set of practices are centered on creating change by encouraging students to take 
ownership of their learning, whereas the second set include pedagogical techniques that may not have 
as great an impact on student engagement. For example, at the higher value-added schools, school 
leaders suggested that quality instruction consisted of student-centered instruction to ensure that 
students could perform independently. In contrast, at the lower value-added schools, there was 
concern that the teachers, not the students, did most of the work and thinking in the classroom with a 
heavy focus on remediation. 

Several stakeholders at the higher value-added schools also named higher-order thinking skills as 
high-quality instruction. Teachers mentioned using questioning strategies or problem solving 
activities (discovery learning, inquiry-based instruction) to reach higher-order thinking skills, 
although most also indicated that mastering this was a continuing struggle. Teachers in the lower 
value-added schools attributed their students’ academic struggles to their lack of background 
knowledge rather than to the quality of instruction. For instance, at Mountainside, teachers questioned 
the feasibility and appropriateness of teaching critical thinking to students with poor educational 
foundations. They also cited student misbehavior as a challenge for rigorous instruction. At Valley, 
teachers also mentioned having difficulty individualizing instruction and directing instruction toward 
the mid- to lower-level students. Teachers primarily relied on tutoring to provide individualized 
instruction and work with struggling students. Interview data from the lower value-added schools 
revealed a lack of understanding of how to foster higher-order thinking skills and what rigor actually 
“looks like” in the classroom. 

Both higher value-added schools seemed to value and encourage cooperative learning, with most 
teachers at Lakeside and several at Riverview mentioning it as a high-quality instructional strategy. 

At the LVA schools, cooperative learning was emphasized in pockets, such as the AVID and ROTC 
programs, but was generally less pervasive than in the higher value-added schools. At Mountainside, 
several teachers reported hesitation to use collaborative activities due to concerns that students’ social 
skills would inhibit group work, and that group-centered assignments might encourage less engaged 
students to rely on one student doing all of the work. 

In general, lower valued-added schools seem to focus more on specific strategies or forms of quality 
instruction such as math practice sheets rather than on the functions of high-quality instruction such 
as helping students become independent learners. 


CLASS-S Data 

To assess the quality of classroom instruction across our four case study schools, we targeted 9 th and 
10 th -grade English/language arts, mathematics, and science classes. Researchers videotaped 
participating teachers’ instruction during the same class period on two consecutive days during the 
first and second visits. Make-up observations occurred during the third visit. Thus, most teachers 
were observed four times. Videographers logged the sequence of activities that occurred in the 
classroom, collected student assignments, and recorded student demographics. 

The CLASS-S rubric was used to code the observations. CLASS-S includes four domains: Emotional 
Support, Organizational Support, Instructional Support, and Student Engagement. The Emotional 
Support domain includes Positive Climate (which reflects the emotional connection and relationships 
among teachers and students), Negative Climate (encompasses the overall level of negativity among 
teachers and students in the class), Teacher Sensitivity (which reflects the teacher’s responsiveness to 
the academic and social/emotional needs and developmental levels of individual students and the 
entire class), and Regard for Adolescent Perspective (which focuses on the extent to which the 
teacher is able to meet and capitalize on the social and developmental needs and goals of adolescents 
by providing opportunities for student autonomy and leadership). The Organizational Support domain 
includes Behavior Management (which focuses on the teacher’s use of effective methods to 
encourage desirable behavior and prevent and redirect misbehavior), Productivity (which considers 
how well the teacher manages time and routines so that instructional time is maximized), and 
Instructional Learning Formats (which focuses ways in which the teacher maximizes student 
engagement). The Instructional Support domain consists of Content Understanding (which refers to 
both the depth of lesson content and the approaches used to help students comprehend the framework, 
key ideas, and procedures), Analysis and Problem Solving (which assesses the degree to which the 
teacher facilitates students’ use of higher-level thinking skills), and Quality of Feedback (which looks 
at the way the teacher’s feedback expands and extends learning and understanding). Definitions of the 
domains and dimensions can be found in Appendix B, along with additional details on the classroom 
observations. 

The CLASS-S data indicate that, across all schools, instructional quality was at a relatively low level, 
particularly in Instructional Support domain. Despite this overall finding, on every domain and 
dimension of the CLASS-S, teachers’ classrooms in higher value-added schools scored better on the 
CLASS-S, with many differences being statistically significant. For example, HVA schools had 
teachers who reached higher levels of analysis and problem solving, focused more on the content 
students were to understand, had higher quality feedback, and more robust instructional dialogue. 
HVA schools also had higher student engagement than the LVA schools. Across all four schools, 
ratings tended to fall in the middle of the CLASS-S rubric, with the exception of the Analysis and 
Problem Solving dimension, which was in the low end, and Negative Climate, which was in the high 
end (representing a lack of negative climate). 
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Table 20. CLASS-S Scores by School 



LVA Schools 


HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

All case 
study 
schools 

Emotional Support 

Positive Climate 

4 40*** 

4.91 

4.93 

4.94 

4.82 

Negative Climate 

6.24 

0 17*** 

6.53** 

6.63*** 

6.36 

Teacher Sensitivity 

434 *** 

4.69 

4.96** 

4.96* 

4.74 

Regard for Adolescent 

2 90 *** 

3.23 

3.49* 

3.43 

3.26 

Perspectives 
Organizational Support 

Behavior Management 

5.05*** 

5.25* 

5 91*** 

5.61 

5.44 

Productivity 

5.07** 

5.15** 

5.68*** 

5.56* 

5.34 

Instructional Learning 

4.33* 

4.40* 

4.76* 

4.79** 

4.55 

Formats 

Instructional Support 

Content Understanding 

4.49 

4.37* 

4.60 

4.68 

4.52 

Analysis and Problem 

2.42** 

2.58 

3.03*** 

2.78 

2.70 

Solving 

Quality of Feedback 

3.85 

3.75* 

4.26*** 

3.85 

3.91 

Instructional Dialogue 

3.18 

3.23 

3.57** 

3.22 

3.30 

Student Engagement 

4.39** 

4.58 

4.83* 

4.88* 

4.67 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean rating compared to the mean from the other schools 
combined. 


Because research on tracking in high schools suggests that higher-track classes tend to have higher- 
quality instruction than lower-track classes, we wanted to assess whether this was occurring in our 
cases study schools, as well as whether higher value-added schools “compressed” the instructional 
quality between their higher- and lower-track classes more than lower value-added schools. Observed 
class periods were selected to ensure a sufficient number of honors and advanced classes were 
included in each school. 

As shown in Table 21, in both higher and lower value-added schools, there were significant 
differences between advanced and regular track classes on all CLASS-S domains. In the higher value- 
added schools, we found significant differences on most dimensions. However, in higher value-added 
schools, there were a few dimensions where we did not find significant differences between tracks: 
quality of feedback, instructional dialogues, and teacher sensitivity, suggesting the HVA schools may 
have compressed variation within the school. On all domains and dimensions, where significant 



differences existed, CLASS-S scores were higher in advanced classes. 


Table 21. Differences in CLASS-S Scores between Advanced and Non-advanced Classes within Each 
Case Study School 



LVA 


HVA 


Mountainside 

Valley 

Lakeside 

Riverview 

Emotional Support 

Positive Climate 

0.38 

0.35* 

0.44* 

0.68*** 

Negative Climate 

0.19 

0.52*** 

0.17 

0.39** 

Teacher Sensitivity 

0.34 

0.26 

0.14 

0.20 

Regard for Adolescent Perspectives 

-0.06 

0.59*** 

0.03 

0.58** 

Classroom Organization 

Behavior Management 

0.25 

0.75*** 

0.80*** 

0.82*** 

Productivity 

0.04 

0.5** 

0.50*** 

0.31 

Instructional Learning Formats 

0.16 

0.48** 

0.53*** 

0.21 

Instructional Support 

Content Understanding 

0.08 

q (}()*** 

0.25 

0.25 

Analysis and Problem Solving 

-0.03 

q 73 *** 

0.60** 

0.32 

Quality of Feedback 

0.12 

0.34 

-0.05 

0.47 

Instructional Dialogues 

-0.17 

q § 4 *** 

-0.07 

0.37 

Student Engagement 

0.52** 

0.61*** 

q 91*** 

0.65** 


* for p<.05, ** for p<.01, and *** for p<.001 denotes a statistically significant between tracks. 


Student Shadowing Logs 

Student shadowing data were collected using a standardized protocol during the first week of May. A 
variety of information about student engagement and interaction was recorded every five minutes 
throughout the school day. Between eight and 10 students were shadowed for a day in each school. 

Data from our student shadowing logs suggest that there were significant differences between our 
higher and lower value-added schools on the majority of indicators. Student engagement varied by 
school, according to the observations, with Valley students more often actively engaged in the 
activity at hand and students in Lakeside most often not engaged. This finding in Lakeside is 
inconsistent with both the fieldwork and CLASS-S data that suggested that adults in the HVA schools 
focused their attention on student engagement. 

There were also significant differences in the frequencies of different types of instructional activities 
that engaged students. Across all the schools, students were most likely to be expected to engage in 
individual work (24%), followed by listening to direct instruction (21%). There were differences 
between schools in the activities in which teachers expected students to be engaged during the 
shadowing observations. Such activities in Mountainside were more to involve whole class discussion 
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and less likely to involve direct instruction. Students in Riverview were less likely to be assigned 
individual work. 


Table 22. Shadowing Results for Four Core Subjects on Expected Classroom Activities 



LVA Schools 

HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

All case 
study 
schools 

Student has an expected task 

96.9% 

97.3% 

92.4%*** 

96.8% 

96.0% 

Engagement 

Actively engaged 

22.4% 

30.0%*** 

19.8%* 

18.3%* 

22 . 2 % 

Passively engaged 

59.3 

54.8*** 

56.3* 

63.7* 

59.0 

Not engaged 

18.3 

15 2 *** 

24.0* 

18.0* 

18.9 

Type of Expected Activity 

Whole class discussion 

19.6%*** 

10.3% 

9.5%* 

11.7% 

13.1% 

Direct instruction 

15.0*** 

16.6* 

23.1 

28.2*** 

21.1 

Group/pair work 

15.3 

23.3** 

13.3* 

17.9 

17.3 

Individual work 

29.7** 

26.3 

29.8* 

14 7 *** 

24.4 

Test/Other academic 

12.9 

13.3 

10 . 1 * 

18.4** 

14.0 

Transition/Non- 

academic/No task 

6.8 

7.3 

13.6*** 

8.7 

9.0 

Other/unclear 

0.8 

3 o*** 

0.6 

0.5 

1.1 


*School estimate has a statistically significant difference from all other schools combined at p<.05 
**School estimate has a statistically significant difference from all other schools combined at p<.01 
***School estimate has a statistically significant difference from all other schools combined at p<.001 
Note: These figures refer to percent of five-minute observational segments. For example, in 22.2% of 
observational segments, the student was observed to be actively engaged. 


Personalized Learning Connections 

Personalized Learning Connections are strong connections between students and adults that allow 
teachers to provide more individual attention to their students . 84 Personalized learning connections 
also refer to developing students’ sense of belonging to at school . 85 Such connections can fall on a 
continuum from strong and robust, leading to connectedness, to weak and non-existent, leading to 
alienation . 86 

Participants across all four case study schools considered building and sustaining strong adult-student 
relationships a priority for fostering student engagement and success; however, actual responses about 
adult-student connections differed. Participants at Lakeside reported extremely positive teacher and 
student chemistry and described the Lakeside Code and Learning Time as the overarching 
mechanisms promoting such connections. (See the Promising Practices section below for full 
descriptions of these practices.) The Code and Learning Time were consistent with the school’s 




dominant focus on academic responsibility — though students also came to hang out with teachers and 
get to know them socially. Essentially, Learning Time provided diverse occasions for personalization 
for academic and social learning — with an evident bias toward academics. At Riverview, there was 
evident leadership in the efforts to meet different students’ needs. The administration at Riverview 
reportedly based employment on teachers’ commitment to activities that would help develop strong 
adult-student relationships. This strategy appeared to pay off as faculty reportedly sponsored many 
clubs and encouraged student involvement. Participants seemed committed to the idea that 
relationships mattered for low-income students who would “go a mile for a teacher.” Yet students 
suggested that the nature of these relationships followed the “school within a school” pattern of 
different treatment for low- and high-achieving students. This might help explain the principal’s goal 
of closing the gap socially as well as academically. 

Valley had a concerted focus on addressing social development needs but only a stated attention to 
academics. Teachers were expected to “do what it takes” to develop relationships with their students, 
including working outside class — and faculty seemed to have solidly bought into this personalization 
goal. Multiple participants, however, noted low levels of rigor at the school. Some teachers seemed to 
emphasize developing relationships with students, rather than making academic demands. At 
Mountainside, only some school personnel were intentional and systematic about building and 
maintaining relationships. The later dissolution of the school’s mentoring program belied 
participants’ view that relationships with kids were a key aspect of practice. Diminished 
personalization practices also seemed to be a function of the school’s significant teacher turnover rate. 
As at Valley, personalization for academic and social learning also involved giving students multiple 
chances because of their difficult personal circumstances. 

All schools appeared to have some measure of academic and social structures to foster personalized 
connections. These structures were “necessary but not sufficient,” however, to meet the needs of low- 
performing students and low-performing schools. At Lakeside, Learning Time, homeroom, and 
Intervention Committee (a student support team that targeted students who were failing) served as 
academic structures that facilitated student-teacher relationships, within the context of cultivating a 
student culture of learning. In many subject areas, very high percentages of the Hispanic students and 
those on subsidized lunches were proficient. Riverside also had several academic structures to benefit 
students: freshman camp, Plato (a standards-based online learning program), JROTC and homeroom. 
However, these structures did not seem integrated into the school culture and instead were either 
dysfunctional or targeted at subgroups. 

Valley organized school personnel (coaches and teachers including grade-level teams) to provide 
formal structures for building personalized connections (via monitoring, home visitation, etc.). 
Support programs at Mountainside included JROTC, athletics, and a community program targeted at 
African American males. However, both lower value-added schools, showed less consistent, systemic 
focus on the academic side of personalization. Valley appeared to do a much better job than its 
counterparts in bringing these structures to bear on the relational side for most students. At 
Mountainside, general failure on both facets of the personalization index may have reflected a failure 
of leadership as well as dysfunctional administrative and disciplinary processes. 
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Responses from participants at the schools indicated that students’ engagement, sense of belonging, 
and/or school pride differed across the four schools. For instance, the level of student involvement in 
extracurricular activities at Lakeside suggested that only a minority of students participated and 
benefitted from these opportunities. This was consistent with interview data, which revealed little 
direct report of students’ sense of belonging or pride, despite high participation in tutoring (Learning 
Time) at Lakeside. At Riverview, student engagement appeared related to students’ academic levels: 
those in higher-level classes reported greater pride than their counterparts in the lower group. 
Participants also reported that a few students were involved in multiple activities, many participated 
in some activities, and about a third did not participate at all — due to family and work obligations 
after school. At Valley, while teachers perceived that many students considered the school a home 
and felt connected to it, teachers also believed school spirit was lacking. This seeming inconsistency 
may have reflected the persistence of the school’s negative image due to its history, despite recent 
efforts to “reinvent” itself. At Mountainside, participants reported that students’ sense of belonging 
arose both from the school’s historical connection to the local community and the students’ lack of 
positive adult relationships outside of school. The interview data suggested that only a small number 
of students participated in extracurricular activities and benefitted from the apparently limited adult- 
student connections. 

Several teacher survey items captured personalized learning connections. These items were broken 
down into three scales. Personalization-Structural Support included items indicating how often 
teachers organized school supports, such as parent-teacher meetings and referrals to community 
organizations, for students who were struggling. Personalization-Extra Help referred to the extent to 
which teachers reported that they or other staff members provided extra help to struggling students. 
Personalization-Social referred to the extent to which teachers knew details about their students’ 
personal lives, such as their academic backgrounds and aspirations, their home life, and identities of 
their friends. Across the district, teachers were most positive about this aspect of personalization. 
While the teacher survey data reflected no statistically significant differences in personalized learning 
connections among the case study schools, Lakeside had the highest average on the Structural 
Support and Extra Help scales, although none of the differences were statistically significant. These 
results were similar to the interview data, which noted a high level of academic supports for students 
at Lakeside. 


Table 23. Teacher Survey Data on Personalized Learning Connections 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Personalization - 
Structural Support 

3.09 

3.11 

3.16 

3.10 

3.08 

(0.45) 

1 -4 

Personalization - Extra 
Help 

2.78 

2.91 

3.03 

2.92 

2.83 

(0.69) 

1 -4 

Personalization - Social 

3.27 

3.18 

3.18 

3.25 

3.25 

(0.72) 

1-5 



* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 

Several items on the student survey also captured personalized learning connections. The 
personalization scale included items that asked students how many adults in the school were willing 
to give extra help with homework, cared about their academic achievement, provided advice about 
graduation requirements, and helped with personal problems. On average, students reported that about 
four to six adults were willing to help them in these ways. The student sense of belonging scale 
ascertained the extent to which students viewed people in the school as a family, felt like they fit with 
the school, and felt that people cared if they were absent. Most students agreed with these items. 
Results for Riverview were significantly lower than the district average on personalization, indicating 
that students had fewer adults they went to for help, and significantly higher on student sense of 
belonging. In general, the similar averages across all four schools supported the interview data 
findings that all the case study schools exhibited aspects of personalization even if the focus differed. 


Table 24. Student Survey Data on Personalized Learning Connections 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Personalization 

2.88 

2.83 

2.85 


2.85 

(1.02) 

1-5 

Student Sense of Belonging 

2.74 

2.74 

2.75 

2.77***(+) 

2.75 

(0.52) 

1-4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


In addition to the above scales, the survey asked students about participation in various school 
programs and the school personnel with whom they discussed various issues, including personal 
issues. The programs most commonly cited across the four schools were tutoring, sports, and 
activities or clubs (see Table 25). Lakeside students were most likely to attended tutoring, which was 
not surprising given the school’s emphasis on Learning Time, and less likely than average to 
participate in sports. At Riverview, community service was the top student activity, consistent with 
what emerged in the fieldwork. Students in Mountainside and Riverview were more likely to 
participate in school leadership, perhaps indicating a stronger presence of a student government 
structure in those schools. 
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Table 25. Student Participation in School Programs 



LVA 

HVA 

District mean 

Mountainside 

Valley 

Lakeside 

Riverview 

Tutoring 

58%*** 

43%* 

75%*** 

26%*** 

44% 

Internship 

2 

1 

2 * 

2** 

1 

Community Service 

18** 

io*** 

13** 

24** 

16 

Sports 

42* 

41 

35*** 

41* 

40 

Activities or Clubs 

30* 

23*** 

26*** 

34 *** 

45 

School Leadership 

12 *** 

5*** 

7 ** 

13*** 

8 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s value compared to the average from the district’s other 12 
schools. 


Most students did not report talking to adults at school about personal or family issues or problems 
(see Table 26), with the exception of conversations with a teacher or counselor about getting a job. 
Students in Mountainside were more likely than students in other schools to talk to a coach about 
personal issues. 


Table 26. Percen tage of Studen ts by School Who Talked to Various Adults about Personal or Family 
Issues or Problems 



LVA 

Mountainside Valley 

HVA 

Lakeside Riverview District mean 

Administrator 

4 %*** 

2 %*** 

2 %*** 

2 %*** 

2 % 

Counselor 

20 *** 

13*** 

28*** 

22 *** 

15 

Teacher 

22 *** 

18 

22 *** 

22 *** 

17 

Coach 

1 2 *** 

7 

5 *** 

6 

6 

Other 

15** 

15*** 

22 *** 

17 

17 

No one 

42 *** 

55* 

47 *** 

go*** 

53 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s values compared to the average from the district’s other 12 
schools. 


Finally, the shadowing data provided evidence about the types of individuals with whom students 
interacted during a typical school day (Table 27). Riverview students were significantly more likely 
to be observed interacting with teachers than students in the other schools. Most notably, in nearly 
half the five-minute shadowing segments, students were not interacting with anyone. These 
observations do not appear to be consistent with other evidence from the case study schools. 



Table 27. Shadowing Results for Four Core Subjects on Who Students Interacted With During Class 



LVA 


HVA 

District 

mean 

Mountainside 

Valley 

Lakeside 

Riverview 

Teacher 

13.2* 

12.0* 

9.8%*** 

27 3*** 

16.5 

Student(s) 

26.4 

24.3 

16.8** 

23.4 

22.9 

Teacher and student(s) 

10.6*** 

6.0 

4.4* 

6.2 

6.9 

No one 

46.5 

54.2 

62.7*** 

39 9*** 

49.7 

Other/unclear 

3.4 

3.7 

6.3* 

3.2 

4.0 


*School estimate has a statistically significant difference from all other schools combined at p<.05 
**School estimate has a statistically significant difference from all other schools combined at p<.01 
***School estimate has a statistically significant difference from all other schools combined at p<.001 
Note: These figures refer to percent of five-minute observational segments. For example, in 16.5% of 
observational segments, the student was observed to be interacting with the teacher. 


Culture of Learning and Professional Behavior 

Culture of Learning and Professional Behavior, the fourth component, refers to students and teachers 
taking part in a strong culture of learning and professional behavior. This culture is defined by a 
shared focus on high expectations for students and emphasis on students’ academic needs among the 
administration, staff and faculty of the school. 87 Students internalize these cultural values as well, 
taking responsibility for their own learning and working together to promote their academic success. 
For teachers, a culture of learning and professional behavior also includes the existence of teacher 
professional learning communities and other communities of practice that define norms of 
engagement, commitment, and heightened professionalism for learning. 88 Several major aspects of 
school climate determine and set the tone of a culture of learning and professional behavior: safety, 
including physical and social-emotional aspects; a culture that supports quality teaching, including 
attention to ongoing teacher professional development, shared high instructional expectations, and 
attention to social and academic learning; and a foundation of effective relationships, including 
respect for diversity and teacher collaboration and connectedness. 89 There was variation across the 
four schools on this component. Although the higher value-added schools had cultures of learning 
based on high achievement, their means of establishing such cultures differed. Professional behaviors 
appeared to vary even more across the schools, with little apparent association to school performance. 
Participants at both higher value-added schools and in Valley referred to a culture of learning that 
reflected a push toward meeting district curriculum standards. Nevertheless, each school manifested 
distinct cultures of learning. Lakeside focused on students developing a sense of responsibility for 
their learning, formally supported by their Code, whereas Riverview had a long-standing tradition of 
excellence with a current emphasis on encouraging lower performing students to enroll in honors or 
AP courses in an effort to close the achievement gap. Also at Riverview, the learning culture 
appeared to be undergirded by more advantaged parents’ use of their social capital to ensure success 
for their children. Even when teachers expressed negativity about these more demanding parents, it 
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was clear that their interest and involvement was preferable to the mostly absent less advantaged 
parents. 


Tutoring requirements and academic expectations at both HVA schools suggested distinct approaches 
to developing a culture of learning. For Lakeside, tutoring was part of the Lakeside Code and was 
seen by teachers as both an academic and social tool. Here, high academic expectations revolved 
around encouraging students to be proactive about their learning with an apparent greater emphasis 
on developing personal autonomy and taking personal responsibility. At Riverview, tutoring appeared 
to be mandated by the administration. While there seemed to be a general pattern of high expectations 
for Riverview students, expectations were particularly high for students in advanced tracks, 
contributing to an achievement gap and the sense of a “school within a school.” 

Both lower value-added schools reflected a pattern of low academic expectations by teachers. At 
Valley, however, this mindset was a byproduct of the administration’s thrust toward building adult- 
student relationships. Mountainside also had a similarly stated goal toward personalization; however, 
it was never realized, due, at least in part, to a significant distrust between the staff and 
administration, high turnover in administrators, and poor or absent disciplinary structures. 

Participants at both schools reported a pattern of giving students multiple second chances to complete 
course work. However, at Valley, struggling students also had to complete mandatory tutoring, 
whereas Mountainside students rarely took up such tutoring opportunities. That said some faculty at 
Valley saw the reduced academic demands as coddling the students. Students’ sense of safety and 
general behavioral compliance were higher at Valley, reflecting a significant turnaround for that 
school. Nevertheless, participants at both schools reported difficulty in motivating students 
academically. It must be noted that when asked about the likely reasons for student success, teachers 
of all schools generally suggested the significant role of a student’s socio-economic status, parental 
involvement, and race. However, teachers in the HVA schools (particularly Lakeside) were more 
likely to discuss these factors as challenges to be addressed, rather than excuses for student failure. 

In all four case study school, teacher collaborations were centered within content departments and 
facilitated by weekly scheduled meetings. Valley also had a strong grade -level teaming structure. 
These meetings formed the basis of formal collaborations that addressed alignment of content for 
those teaching the same courses. Efforts at vertical alignment were also mentioned as a basis for 
formal collaboration; however this process seemed more formalized at Riverview and, in some 
subject areas, Valley. Informal collaborations among faculty members to address instructional needs 
were apparent in Lakeside, where teachers reported e-mailing each other and “sharing ideas,” and in 
Valley, where there was frequent sharing of resources. At both Lakeside and Valley, participants also 
reported talking about students’ academic and social issues. The apparent lack of informal sharing by 
faculty at Riverview and Mountainside may be related to each school’s professional climate. At 
Riverview, there was a sense of competition among faculty in advanced tracks, perhaps as a result of 
the school’s historically high achievement focus. At Mountainside, there was an apparent sense of 
instability and lack of trust, with high personnel turnover rates. 

Professional development (PD) opportunities were generally either school-sponsored or district-led. 
District-led professional development was available to all four schools, but attitudes toward these 



programs differed. Professional development opportunities at Lakeside were widely available, 
especially through informal interactions among colleagues, such as common planning, walkthroughs, 
coaching for lead content teachers (LCTs), and departmental meetings. Training in the Kagan method 
for cooperative learning was pervasive as a means of instructional improvement, particularly in 
regards to cooperative learning. District-sponsored programs/activities were the major source of 
professional development at Riverview. At Mountainside, participants reported various occasions for 
professional development, although many questioned the validity and helpfulness of feedback after 
classroom observations. Many also questioned the effectiveness of mandatory PD (e.g., Kagan 
strategies). Valley participants tended to consider professional development externally driven from 
the district, and not based on site-specific tasks. Yet, this Paining at Valley was deemed more 
effective than site-based PD, which was based on principal-initiated book clubs. It appeared that the 
openness to external PD — as in the case of Lakeside and Valley — may also have been related to the 
proportion of new teachers on staff and an assumption they needed such opportunities. As such, while 
adults at Riverview reported dissatisfaction or displeasure, those at Lakeside and Valley reflected 
appreciation of district-led PD. In the case of in-house PD at Mountainside and Valley, one factor that 
may have affected teacher attitudes was whether they felt that they or their teaching methods were 
being unduly criticized. In Lakeside and Mountainside, the role of LCTs in providing PD to match 
teachers’ specific needs was seen as beneficial. 

The teacher survey also provided data on evidence of culture of learning and professional behavior. 
The scale that measures the degree to which collaboration improves instruction captured the extent to 
which teachers planned lessons together, reviewed student work and data with other teachers, 
observed each other, and worked to identify individual students’ needs together. Teachers at 
Lakeside, Riverview, and Valley had significantly higher averages on this scale. The scale measuring 
frequency of collaboration indicated that teachers collaborated one to three times a month on lesson 
plans and reviewing students’ work and data, with most teachers agreeing that they have a planning 
period with other teachers in their department/subject area. Teachers at Riverview and Valley 
reported more frequent collaboration (i.e., planning lessons, reviewing student work together) than 
the district average. The scale measuring frequency of peer observation and feedback indicated that 
most teachers observed each other and provided feedback less than once per month. None of the 
schools were statistically different from the district average on this measure. The scale measuring 
positive faculty climate included items such as whether administrators respected teacher opinions and 
whether teachers enjoyed working in the school, had opportunities to do their best every day, and 
recognition were recognized for a job well done. Teachers at Lakeside and Valley were significantly 
more likely to report positive faculty culture, and teachers at Mountainside were significantly less 
likely to report positive faculty culture. This was consistent with the interview data. In terms of 
teacher perceptions of the student climate, the only measure available is about bullying/school safety, 
on which Lakeside teachers reported significantly less bullying compared to other teachers in the 
district. 
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Table 28. Teacher Survey Data on Culture of Learning and Professional Behavior 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Collaboration Improves 
Instruction 

3.01 

3.14**(+) 

3.21*(+) 

3.08*(+) 

2.90 

(0.75) 

1 -4 

Frequency of Peer 
Observation and 
Feedback 

2.44 

2.39 

2.23 

2.30 

2.29 

(0.80) 

1 -4 

Frequency of 
Collaboration 

2.51 

3.01*(+) 

2.68 

2.94*(+) 

2.76 

(0.83) 

1 -4 

Positive Faculty Climate 

2.34***(-) 

3.28***(+) 

3.27*(+) 

3.12 

3.03 

(0.61) 

1 -4 

Bullying/School safety 

2.68 

2.23 

3.20*(-) 

3.40 

2.36 

(0.88) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the 
district’s other 12 schools. 


Several items on the student survey data captured aspects of school climate, which falls under this 
component. While there were differences among the schools, no clear pattern emerged between the 
HVA and LVA schools as a group (see table 29). Valley had high scores on the negative school 
culture construct, suggesting students in that school thought their peers felt it was OK to cheat and 
make racist or sexist comments. However, students in Valley also had higher scores on a general 
measure of school climate, which captures the extent to which students feel respected, get recognition 
for doing well, and are encouraged by the teacher to share ideas. Somewhat surprising given the 
interview data, Riverview students’ survey responses were less favorable on school climate, but also 
less negative on indicators of school culture, safety, and bullying. Students in Valley and Lakeside 
reported more bullying. This differed from reports by Lakeside teachers of less bullying than the 
district average. Students at Mountainside reported feeling less safe, which was consistent with the 
interview data. 



Table 29. Student Survey Data on Culture of Learning and Professional Behavior 


LVA HVA 


District 

Mean Scale 



Mountainside 

Valley 

Lakeside 

Riverview 

(SD) 

Range 

Student Responsibility: 
Negative School Culture 

3.48 

3.68***(+) 

3.51 

3.41**(-) 

3.48 

(1.14) 

1-5 

School Safety 

3.1 1*(-) 

3.20 

3.19 

3.22***(+) 

3.18 

(0.56) 

1-4 

School Climate 

2.89 

2.96***(+) 

2.87 

2.87**(-) 

2.90 

(0.46) 

1-4 

Bullying 

2.40***(-) 

2.65***(+) 

2.66***(+) 

2.48***(-) 

2.56 

(0.79) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Parent survey data also provided insight on school climate. The climate scale captures whether 
parents feel they are treated with respect at the school, feel welcome there, feel it is a safe place, and 
feel it is preparing their child to succeed. Parents at Riverview were significantly more positive 
regarding the school climate than parents from other schools in the district, while parents at 
Mountainside were significantly more negative. The negative climate rating by parents of 
Mountainside students was consistent with the data from teacher surveys and interviews. The bullying 
items asked parents to indicate whether their child had been bullied at school or via texting, Internet, 
or phone. On average, parents across all four schools indicated that their students had been bullied, 
though none differed significantly from other parents in the district. 
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Table 30. Parent Survey Data on Culture of Learning and Professional Behavior 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

School Climate 

2.86*(-) 

2.99 

3.09 

3.21***(+) 

3.02 

(0.67) 

1-5 

Bullying 

3.37 

3.36 

3.34 

3.43 

3.36 

(0.79) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


The administrative data included several indicators that helped us understand the culture of learning 
for students (Table 31). The four case study schools had similar student attendance rates, although 
rates at the HVA schools had risen in recent years and both have slightly higher rates than the LVA 
schools. The HVA schools also had fewer students with disciplinary action, suggesting fewer 
behavioral concerns. Riverview had seen a marked increase in disciplinary referrals. 


Table 31. Attendance and Disciplinary Behavior for Most Recent Three Years and Change over 
Recent Two Years 



LVA Schools 

HVA Schools 

District 

Mean 


Mountainside 

Valley 

Lakeside 

Riverview 

Most recent 3 years 
Attendance Rate 

90% 

92% 

94% 

93% 

92% 

S. w / "Disciplinary Action" 

47 

42 

22 

22 

31 

Two-year change 
Attendance Rate 

-5% 

-1% 

16% 

6% 

4% 

S. w / "Disciplinary Action" 

-2 

24 

6 

11 

9 


Note: These changes are from SY08 to SY10, rather than SY09 to SY1 1 

* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s value compared to the average from the district’s other 12 
schools. 


Connections to External Communities 

Connections to External Communities refers to robust connections and relationships between schools, 
families, and other community partners. The literature on high schools and parent and community 
relationships is limited, especially when compared to the vast conceptual and empirical literatures on 
parental and community engagement in elementary schools and in education in general. While there 
is agreement with the notion that "families, communities and schools hold shared and overlapping 





responsibility for the healthy development and the social and academic success of all children,” 90 less 
is understood as to how these aspirations are fulfilled in high school. To the extent that the literature 
on the relationships between high schools and student achievement does address external 
constituencies, the focus is primarily on parents, with much less attention to the larger community in 
terms of social agencies, businesses and community assets. Furthermore, the empirical research in 
high schools is clear: parental support and parent involvement matter, as these provide sources of 
social capital. 91 

Outreach to External Communities 

Generally speaking, there were few systematic differences between the HVA and LVA schools in 
outreach to external communities. Riverview set itself apart as an outlier in the sense that the school 
leaders had made systematic attempts to involve parents. These efforts had largely succeeded. Parents 
of both segments of the school population (the higher performing 60 percent and the lower 
performing 40 percent, which was majority Latino) were involved in the school. However, teachers 
still reported difficulties engaging the parents of students in lower-track classes. The other schools 
notably made somewhat systematic attempts to reach out, but the efforts did not stir a big response. 
School participants cited the main constraints in fostering successful school-community linkages as: 
low parental educational attainment, language barriers, and unease with the schooling institution. All 
four schools leveraged connections to the community for resources, though to varying degrees, to 
provide opportunities for students and to improve community relations or the school’s image in the 
community. Mountainside and Valley both had relationships with companies that clearly supported 
academics in the programs of choice. Community support opportunities for students seemed best at 
Riverview, where students benefitted from a partnership with a local four-year university. Most of 
these school outreach programs involved and targeted small numbers of individuals were short-run, 
and were not directly tied to academic goals. This was true even of programs considered most 
successful, such as blood drives or Valley’s “Pyramid Fair,” an event informing families about the 
elementary, middle and high schools in a given school zone. 

Parent Involvement 

Teachers and staff in both LVA and HVA schools regarded parent involvement as a significant 
challenge and an area for improvement. Only Riverview had a fully functioning PTO, which was 
described in superlatives as the largest, strongest that teachers had seen. The lack of functioning PTOs 
at the other three schools may have reflected the amount of parent involvement there. At Riverview, 
the parents of socioeconomically advantaged students were involved (some teachers suggested over- 
involved), while the school also struggled to connect with the parents of lower-class and Hispanic 
students. The school had several efforts underway to engage Hispanic families more. In the LVA 
schools, Lakeside and the Hispanic portion of Riverview, family characteristics that can hinder 
academic success (single-parent homes, lack of English proficiency, low parental educational 
attainment) were cited as impediments to successful school-family linkages. Parental engagement was 
perceived as particularly weak in Mountainside and Valley. 

Systematic differences emerged in how HVA and LVA schools approached the process of involving 
parents. In both HVA schools, participants reported systemic, school-wide programs and practices 
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targeted at engaging parents. In Riverview, these efforts were more proactive - especially in terms of 
reaching out to Hispanic parents. For example, a concerted effort was made to hire Spanish-speaking 
staff, to have these staff at events with Hispanic students, and to make special efforts to get Hispanic 
parents to attend school events. In Lakeside, efforts to involve parents were reportedly more reactive; 
parents were engaged in response to students’ performance and behavioral issues. At Valley, teachers 
were required to contact all parents in the first six weeks of school. In Mountainside, participants 
reported that efforts to engage parents were more focused on individual efforts than systemic and 
school wide. In Mountainside, for example, the task of forming relationships with and engaging 
parents was undertaken by social-emotional support personnel such as the Umoja 92 coordinator or the 
stay-in-school-coordinator. Interviews with district officials suggested that the HVA schools involved 
parents more successfully. 

Quantitative Data 

The teacher survey data supported the qualitative data’s finding that connections to external 
communities were highest in Riverview and lowest in Mountainside (Table 32). Parent outreach- 
school support refers to the extent to which teachers reported that the school as a whole encouraged 
and supported parent involvement. 


Table 32. Teacher Survey Data on Connections to External Community 



LVA 


HVA 

District 






Mean 

Scale 


Mountainside 

Valley 

Lakeside Riverview 

(SD) 

Range 

Parent Outreach - School 
Support 

2.54 

2.65 

2.56 2.89**(+) 

2.71 

(0.61) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


On the student survey, the scales with the most variation across schools contained questions relevant 
to parents, grouped here under this component (Table 33). The parent press for academic achievement 
scale indicated that, on average across the district, parents talked with students two to three times a 
semester about how they were doing in school, what they were studying in class, and their homework 
assignments. The parent connections-involvement scale indicated that, in general, students reported 
that parents helped with homework assignments, volunteered at school, and helped them decide what 
classes to take two to three times per year, and indicated that their parents did not know most of their 
teachers. Parent connections-future orientation refers to the extent to which parents encouraged 
students academically and talked to them about college and future career plans. On average, students 
report this occurring two to three times per semester. Consistent with the interview data, Riverview 
scored at the top of the district on all parent-related constructs. Also consistent with the fieldwork 
data, HVA Lakeside and LVA Valley scored at the bottom of the district on most of the parent-related 
constructs. Mountainside fell near the district average on these constructs, although it was higher on 


parent connections-involvement, which was not consistent with what interview and focus group 
participants reported. 


Table 33. Student Survey Data on Connections to External Community 



LVA 

HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Parent Press toward 
Academic 
Achievement 

2.99 

2.80***(-) 

2.81***(-) 

3.23***(+) 

2.98 

(0.97) 

1 -4 

Parent Connections: 
Involvement 

2.24***(+) 

1.85***(-) 

-1.90***(-) 

2.32***(+) 

2.08 

(0.72) 

1 -4 

Parent Connections: 
Future Orientation 

3.17 

3.03***(-) 

3.07**(-) 

3.32***(+) 

3.17 

(0.88) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Similar to the teacher and student survey data, the parent survey data indicated that parents were most 
connected at Riverview and less connected at the LVA schools. On the parent survey, the scales 
measuring Connections to External Communities included parental involvement, school-home 
communication, learning at home, and parental self-efficacy . The school-home communication 
scale included items on the occurrence and timeliness of communication regarding a child’s academic 
work and behavior and school events and activities. The self-efficacy scale captured parents’ 
estimates of how quickly they responded to teachers’ requests, whether they felt their involvement 
was appreciated by teachers and staff, and whether they believed parents were responsible for helping 
their child succeed academically. Generally, parents agreed that their child’s school communicated 
with them regularly and felt efficacious when it came to parenting. The learning at home scale 
captured the extent to which parents felt that the school provided them with clear information on how 
to interpret assessments and reports and to monitor and discuss school at home. It also gauged the 
degree to which they felt able to help their child with schoolwork. Parents at Lakeside, Mountainside, 
and Valley were more likely to disagree with these statements than parents at Riverview. The parental 
involvement scale captured the levels of parental involvement in activities such as parent-teacher 
conferences, academic events, extracurricular activities, and help with homework. Parents at 
Riverview were mostly likely to report some involvement, while parents at Valley were mostly likely 
to report low involvement. 
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Table 34. Parent Survey Data on Connections to External Community 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

School-Home 

Communication 

2.82 

2.84 

2.85 

3.12***(+) 

2.94 

(0.77) 

1 -4 

Self-efficacy 

3.01***(-) 

3.00**(-) 

3.17 

3.38***(+) 

3.23 

(.75) 

3.23 

Learning at Home 

2.81 

2.67*(-) 

2.86 

3.05***(+) 

2.85 

(0.88) 

1 -4 

Parental Involvement 

2.35*(-) 

2.14***(_) 

2.34 

2.78***(+) 

2.52 

(0.75) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Systematic Performance Accountability 

Systemic Performance Accountability refers to the “new accountability” espoused by recent education 
reforms, in which outcomes take precedence over processes in the evaluation of scholastic 
performance. 94 The literature on systemic performance accountability in secondary schools finds that 
efforts to shift the focus of accountability from educator processes to student learning outcomes do 
not always achieve the desired effects on either processes or outcomes. However, the success or 
failure of an accountability policy appears to rely less on the quality of the policy’ s design than on the 
quality of its implementation. 95 One finding consistent across the literature is that the success of 
accountability policies, as measured by either implementation fidelity or student achievement, is 
mediated by teachers’ beliefs about their students. 96 Specifically, whether teachers alter their practices 
in response to new policies hinges on educators’ willingness to acknowledge connections between 
instructional practices and student learning (personal responsibility) and between student learning and 
the policy’s outcomes of consequence (data validity). Acknowledging these linkages determines 
whether educators respond to accountability measures with instructional strategies or deflection 
strategies. 

Based on interview data, there appear to be a few areas of convergence/divergence among our case 
study schools regarding systemic performance accountability: the nature of the culture of 
accountability in each school, trust among actors in the system, and the presence of consequences for 
performance. Across all three of these constructs, there were no systematic differences between HVA 
and LVA schools. Instead, we found qualitative differences between individual schools - in particular 
Lakeside and Mountainside. Regarding school culture of accountability, for example, participants in 
Lakeside reported a strong, proactive culture of high expectations and accountability. This was 
reinforced by the Lakeside Code, and it encouraged internalization of responsibility and 
accountability among actors. In Mountainside, by contrast, participants depicted a far weaker culture 



of accountability, in which discipline was inconsistent and actors (especially students) might not 
internalize responsibility or accountability for their performance. In Riverview and Valley, views on 
cultures of accountability appeared to be more mixed, with actors indicating both strengths and 
weaknesses. 

Trust among school personnel disaggregated similarly. While no consistent differences emerged 
between HVA and LVA schools, Lakeside and Mountainside appeared to represent poles between 
which Riverview and Valley fell. Participants in Lakeside reported a strong sense of trust between the 
administration, faculty, and students; participants reported that this foundation of trust created an 
environment in which teachers were willing to accept performance feedback as well as consequences 
for poor performance. In Mountainside, participants reported relatively weak trust among school 
personnel. As a result, teachers may have perceived feedback to be invalid (or, in some cases, ill- 
intentioned), or that consequences/discipline were unfairly applied. In Valley, participants reportedly 
perceived a general sense of trust, while interviews at Riverview produced mixed reports on teachers’ 
perceptions of the validity of feedback. 

Finally, there did not appear to be consistent differences between HVA and LVA schools on the 
application of consequences for students. Lakeside participants reported a strong perception of 
consequences for failing to meet standards - or rewards, conversely, for exceeding accountability 
standards. Interviews at the other three schools revealed more mixed perceptions of the nature of 
consequences and their application. In both LVA schools participants reported a culture of multiple 
chances allowing students several opportunities to make up for failure. These practices had both 
positive and negative aspects in respondents’ views. Such limited expectations of student 
accountability in the LVA schools supported the finding that academic press was a key difference 
between HVA and LVA schools. 

Data from the teacher surveys indicated that scales related to performance accountability (Teacher 
Accountability and Principal Effectiveness in Supporting Systematic Performance Accountability) 
were particularly high in Lakeside and particularly low in Mountainside, supporting many assertions 
drawn from the qualitative data. The Teacher Accountability construct measures the extent to which 
teachers feel that there are consequences for teachers who don’t perform well, that feedback on their 
instruction is valuable, that other teachers hold them accountable for performance, and that teachers 
who don’t do well are given opportunities to improve. The Principal Effectiveness at Supporting 
Systemic Performance Accountability construct captures the teacher’ s evaluation of the principal in 
holding faculty accountable, advocating for holding students accountable for achieving at high levels, 
and challenging faculty who do not hold students accountable. Both HVA schools scored higher than 
their LVA counterparts on the Teacher Accountability scale. Mountainside had the lowest mean 
scores across both systemic accountability-related scales, supporting many of our findings from the 
qualitative data. Scores on the items measuring principal support for systematic performance 
accountability were mixed, with Valley scoring higher than the district average and Riverview similar 
to it. The high regard teachers in Valley had for the principal may have contributed to this high rating. 
Somewhat to our surprise, given the prior findings about observation and feedback, none of the case 
study schools were statistically different from district averages perceptions about on the Instructional 
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Support from Leadership measure, which captures how frequently teachers are observed and get 
feedback from administrators on their instruction. 


Table 35. Teacher Survey Data on Systemic Performance Accountability 



LVA 


HVA 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

Mean 

(SD) 

Scale 

Range 

Instructional Support from 
Leadership 

2.10 

2.46 

2.56 

2.33 

2.33 

(0.77) 

1 -4 

Teacher Accountability 

2.45***(-) 

3.08 

3.22*(+) 

3.16**(+) 

2.98 

(0.60) 

1-4 

Principal Effectiveness at 
Supporting Systemic 
Performance 
Accountability 

2.20***(-) 

3.45***(+) 

3.53***(+) 

3.21 

3.07 

(0.82) 

1-4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Systematic Use of Data 

Systemic Use of Data refers to data use or data-based decision-making as a practice critical to school 
improvement efforts. Yet it would be faulty to assume that access to data alone will lead to more 
effective practice. Rather, research on systematic data use suggests that effective practice requires a 
critical consideration of both which data and which forms of use are most effective in improving 
academic performance. Though research specific to data use in high schools is scant, a consistent 
finding across this work is that where data use is effective, the power to make data-based decisions is 
diffuse, collaborative, and pervasively integrated into practice. In contrast, data-based decisions made 
centrally and dictated to teachers breed resistance, foster mistrust, and do not improve instructional 
practices. 

Based on interview data, participants across our case study schools tended to discuss their use of data 
in terms of three primary elements: access and availability, capacity and use, and cultures of data use. 
There did not appear to be systematic differences between our HVA and LVA schools with regard to 
this component. Across all four schools, for instance, participants reported that they had access to a 
variety of performance data. Similarly, in all four schools, much of the data referenced by participants 
were derived from external assessments, such as the TAKS or Curriculum Based Assessments, which 
are designed by the district and administered every six or nine weeks, depending on subject area. 
Mountainside deviated from the rest to a certain extent in that multiple participants reported using 
some assessments developed by teachers within the school to inform their practice (e.g. bell-ringer 
quizzes — or quizzes to be taken at the beginning of class — and other diagnostic assessments in 
math). Administrators across schools also reported accessing other types of data, including classroom 



observation scores, attendance data, and grades. 

Similarly, there do not appear to be systematic differences in how our HVA and LVA study schools 
used data. Participants in each school reported using it primarily to inform instruction by guiding 
pacing, identifying areas of student deficiency, and informing teachers’ efforts to design lessons. 
Similarly, participants across all four schools also reported using data to assign students to classes. 
One potential area of divergence was the use of data for teacher assignment, with Lakeside and, to a 
lesser extent. Valley, reporting that they used data on teachers’ performance when assigning teachers 
to courses (although the nature of performance data used were not made explicit). 

Finally, there are no systematic differences between HVA and LVA schools in having cultures of data 
use. Across all four schools, participants reported largely functional cultures of data use. As such, 
data-driven decision-making was a necessary part of their work, and participants reported often 
working collaboratively to analyze and make use of performance data. Valley’s administrative leaders 
may have emphasized data-use more heavily, reportedly a push by the current principal. Responses 
indicated that data may have been used more pervasively across levels (administrative, instructional, 
etc.) at Lakeside than in the other study schools. 

Teacher survey responses on data use were relatively mixed, and did not indicate that different data 
use practices drove student growth in our higher value-added schools (Table 36). The district mean 
for the Data Use scale indicates that teachers generally agreed that they are using data from 
Curriculum-Based Assessments (CBAs) and other sources to enhance instruction, plan lessons, set 
learning goals, form small groups for targeted instruction, and identify students for tutoring or other 
interventions. Survey results revealed higher mean scores on the data use scale for both LVA schools 
than both HVA counterparts (although only Valley’s results were statistically different from the 
district average). Some evidence in the interviews indicated, however, that accountability frameworks 
and grant programs at our LVA schools may have been encouraging a relative emphasis on 
incorporating data into decision-making processes. Further work could focus on unraveling the 
“chicken or the egg” question implied by this, to determine whether an increased focus on data-driven 
decision-making pre-dated, or followed, identified poor performance. 
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Table 36. Teacher Survey Data on Data Use 



LVA 

HVA 

District 





Mean 

Scale 


Mountainside Valley 

Lakeside Riverview 

(SD) 

Range 

Data Use 

3.15 3.18* (+) 

3.10 2.93 

3.01 

(0.61) 

1 -4 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean scale rating compared to the mean from the district’s 
other 12 schools. 


Organization of the Learning Environment 

While organization of the learning environment was not one of the original research-based essential 
components, we did want to understand how key organizational functions of the school took place. 
These included how students were assigned to courses, how teachers were assigned to courses, and 
the assignment of the leadership team. 

Student Assignment to Courses 

The interview data described the contribution of multiple stakeholders to student course assignment. 
Overall, student assignment to courses was similar at all schools. However, schools differed in the 
amount of student and parent input they allowed in the process of course selection and course rigor. 

At all schools, the counselors assigned students to courses, considering test data, previous courses 
taken, and graduation requirements. At all schools, students provided some limited input about the 
classes and level of rigor they preferred. However, students had considerably more autonomy in 
course selection at Valley, which allowed students to select classes but required teachers and 
counselors to sign off on the level of rigor. All schools allowed parents to provide input on the rigor 
of classes students were assigned to. At Lakeside, honors courses had open enrollment. Riverview 
encouraged students to take advanced courses, usually when teachers and counselors believed a 
student could succeed at higher levels. In contrast, at Mountainside, a student’s course rigor usually 
remained static over the course of his or her schooling. 

Student survey data indicate that students at all four schools talked to various adults about what 
courses to take (see Table 37). The survey data support the interview data, revealing that students 
talked to counselors more than any other adult in school about course selection. Students at 
Mountainside reported the lowest percentage of students talking to counselors about what courses to 
take, while Riverview reported the highest. At all schools, teachers were the next adult students 
reported speaking with about course selection. Similar percentages of students at each school 
responded “no one” on this measure, which was the third most common response at all schools except 
at Mountainside where it tied with “other”. 


Table 37. Number and Percen tage of Students by School Who Talked to Various Adults about What 
Courses to Take 



LVA Schools 

HVA Schools 

District mean 

Mountainside 

Valley 

Lakeside 

Riverview 

Administrator 


3% 

3% 

3%*** 

6 % 

Counselor 

63*** 

65*** 

68 

59 ** 

68 

Teacher 

28 

30 

29 

22 *** 

29 

Coach 


9 *** 

g** 

8 

7 

Other 

24 *** 

10 * 

24 *** 

12 

12 

No one 

13 

16** 

13* 

15* 

14 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s value compared to the average from the district’s other 12 
schools. 


Teacher Assignment to Courses 

The process by which teachers received course assignments differed at each school. Interview data 
revealed that teachers at Lakeside, Mountainside, and Valley provided input about course preferences. 
Lakeside considered these preferences, along with student needs and input from the Lead Content 
Teachers and other administrators, when assigning teachers. Decisions about what level of rigor a 
teacher should instruct were informed by an administrator’s classroom observations and data on AP 
pass rates for teachers. At Mountainside, the school considered teacher input, along with the areas in 
which teachers held state certification, with very little evidence that the process was data-driven. At 
Valley, the leadership team considered teachers’ preferences, along with their energy level, prior 
performance, and their likely fit with other teachers in the academic department. At Riverview, the 
leadership team and the department chairs assigned teachers based on departmental needs. In nearly 
all cases, teachers of core academic subjects taught either advanced-or on-level courses; most did not 
teach across the tracks. 

Assignment of Leadership 

Assistant principals at all four schools oversaw one or more departments, with assignments based on 
varying criteria. At Riverview, each assistant principal oversaw students whose surnames fell within a 
certain a section of the alphabet, while at Lakeside, the assistant principals received a grade-level 
assignment. Mountainside combined the two approaches with one assistant principal overseeing 9 th 
grade and the other two splitting the rest of the grades in alphabetic groups. Valley had a unique 
approach among the study schools in that assistant principals received only departmental assignments. 
Each person was assigned to handle all student discipline issues on certain days, and then relieved of 
this duty on other days to focus on other concerns — including instructional leadership. Further, there 
is not a consistent difference between HVA and LVA schools in whether administrators or counselors 
loop 9X with students. 
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Promising Practices. 


We provide seven promising practices to describe specific programs and approaches that appeared to 
contribute to increasing students’ responsibility for their own learning, and describe how the success 
of the practices was sustained by the school-wide facilitating conditions evident in effective schools. 
The first four practices were in the two higher value-added schools. The next three were in the two 
lower value-added schools. We stress that effective practices can be found in both types of schools. 
One key difference between higher and lower value-added schools was the degree to which such 
practices existed. In the higher value-added schools such practices were more systemic, spanning the 
entire school, while they existed only in portions or pockets of the lower value-added ones. Because 
the examples are written to illustrate how schools used the practices to enact the school-wide 
facilitating conditions, key terms related to the conditions appear in bold type. 


1. Lakeside: The Code 

Description 

The Lakeside Code can be summed up by the school motto Effort Required. The cornerstone of 
Lakeside's improvement efforts, it included seven behaviors students were required to demonstrate 
to increase their own accountability and responsibility for learning: 1) attend school and be on time; 

2) come prepared to class and take advantage of tutoring opportunities during Learning Time; 3) find 
out required assignments after missing school; 4) be able to either explain what the teacher has 
emphasized or have a question about what isn’t clear; 5) practice independent applications of material 
to ensure understanding and attend Learning Time when you don’t understand; 6) talk to teachers 
about assignments and tests where you struggled; and 7) monitor your own progress through 
Assignment Logs. Staff members were expected to know these elements of the Code, and school 
personnel occasionally described it as a student code of conduct and treated it as such by linking 
violations to sanctions such as referral to an Intervention Committee and in-school suspension. 
However, the Code also set expectations for academic and instructional behaviors of both students 
and teachers. Each element included specific behaviors required for compliance by students, as well 
as expectations for teachers to support students in taking responsibility for their own learning. For 
example, point 4 expected students to pay attention and ask questions in class, while expecting 
teachers to press students for explanations. Teachers were expected to randomly select students to 
respond, teach students how to manage their attention, and ask students to summarize their learning 
from the previous day. 

Although the Code only took effect in the 201 1-2012 school year, it followed years of work by school 
leaders to develop a shared understanding of the need to reduce student dependence on teachers and 
build student responsibility in order to improve student learning. The principal described how a core 
group of faculty and administrators compared the school’s student achievement data with that of other 
schools in the district and determined that what successes they had achieved had come from putting 
the locus of responsibility on teachers. This group realized that students needed to take more 


responsibility if the school was to improve further. The current principal, while new to the role in 
201 1-12, was part of the leadership team during these discussions. Thus the Code was, in a sense, a 
formalization of this shared vision and a structure to support its enactment. The Code initially focused 
on students, but as it was implemented, the school realized that students were more likely to meet the 
expectations if the Code included expectations for teachers, too, and these were added midyear. For 
example, point 7 of the original Code required students to monitor their progress with the Assignment 
Log. The Code evolved to require teachers to help students by training them to use the Assignment 
Log and calculate their grades, giving them time to update their logs after returning work, monitoring 
the logs, and refraining from posting grades. 

The Code was posted in almost every classroom and teachers (with the encouragement of leadership) 
referred to elements of the Code during class. For example, students who didn’t bring materials to 
class were reminded that the Code expected them to come prepared. Teachers reported monitoring 
Assignment Logs and rewarding students whose logs were accurate, while administrators randomly 
asked students to share their logs. School leaders supported implementation of the Code and 
mentioned it frequently in various types of interactions with teachers (e.g., weekly principal e-mail to 
the faculty, as feedback after classroom walk-throughs, in faculty meetings). The principal also 
provided professional development opportunities to teachers to help them learn how to implement 
elements of the Code. 

Part of what made the Code effective was its integration and alignment with other systems in 
Lakeside. The remainder of this case example focuses on how the Code was enacted through the 
strategies used by the school and facilitated by several school-wide facilitating conditions. In 
particular, points 1,5, and 6 of the Code had explicit expectations for when students should attend 
Learning Time and be referred to the Intervention Committee for non-compliance with the Code. 

How the Lakeside Code Supported Increasing Student Ownership and Responsibility 

The Lakeside Code encouraged students to take responsibility for their own learning by explicitly 
teaching behaviors that demonstrate personal investment in learning. The school also sought to help 
students develop a sense of efficacy by recognizing that they have the ability to achieve if they exert 
the effort required to succeed. 

The enactment of the Lakeside Code illustrates how the teachers and other adults in the school used 
academic press and academic support to scaffold students’ learning of both academic and social 
behaviors to guide them in assuming ownership and responsibility for their own learning. Further, the 
structures and practices that contributed to academic press and academic support were integrated and 
aligned with each other. In terms of academic press, Lakeside created shared practices around 
student accountability, high expectations, and an academically focused environment, within an 
instructional vision devoted to student ownership, responsibility, and engagement. There were clear 
expectations for students with explicit consequences for not following the Code. Teachers also 
reported that they were supported in maintaining high expectations for students and holding them 
accountable for meeting those expectations. To create an academic environment of high expectations, 
the school explicitly set the goal for student performance higher than that of the district as a whole. 
The Code also set out expectations for student conduct, but it focused more on academic and 
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instructional behaviors than discipline or social behaviors. Notably, the school decided to de- 
emphasize the dress code to focus student accountability on elements in the Code. Finally, the explicit 
expectations for all teachers outlined in the Code made it clear that there are instructional 
implications. The school’s primary improvement strategy, then, sought to help students learn before 
they had a chance to fail and require remediation. 

Academic support was conveyed through school-wide mechanisms to help students meet the 
expectations set out in the Code. One mechanism was the Learning Time tutorial system (see below), 
which designated a time when teachers were expected to be available and students were expected to 
attend if they had missed assignments and/or were struggling in class. Thus, Learning Time was 
another example of a school-wide effort to teach students to take responsibility for their learning by 
expecting behaviors that demonstrate personal investment in their learning. No one could plead 
ignorance of the Code’s requirements. Teachers and other school personnel posted it throughout the 
building, including in every classroom, and created an Intervention Committee process (described 
below) to work with students who lapsed. 

Several school-wide facilitating conditions supported enactment of the Code. First, as the centerpiece 
of Lakeside’s improvement efforts, the Code codified the school’s shared vision and the aligned 
and coherent structures created to support the vision. As noted above, prior to formalizing the 
Code, the school leaders developed a shared understanding that they needed to reduce student 
dependence on teachers and make it clear students were responsible for their own learning. Over 
several years, most adults in the school began to buy into this developing goal. Through this 
evolution, Lakeside became mission-driven. Clear processes and procedures were put in place to meet 
the goal and achieve the shared vision. Further, policies, practices, and activities were aligned to each 
other and to the vision. One reason the Code was effective is that, as the most prominent embodiment 
of the school’ s vision, it set forth both specific behaviors — and interventions for infractions — that 
were integrated into and aligned with other school-wide processes. Thus, it represented a holistic 
vision that not only codified the explicit expectations of Learning Time, Assignment Logs, and the 
Intervention Committee, but it fostered a professional environment in which in-service training could 
be linked to the Code and the implementation of Code provisions could be monitored through regular 
classroom walk-throughs. 

Lakeside also had a culture of trust that was fostered by communication between administrators and 
teachers. This trust facilitated much of the shared school practices as it allowed administrators and 
department heads to hold teachers to high expectations without significant resistance. In particular, it 
increased teacher buy-in to the shared vision and allowed teachers to feel supported when holding 
students accountable and pressing them to be responsible for their learning. 

The relative stability of leadership, teachers, and students facilitated school practices and the 
development of the shared vision and understanding of the Code. Even with a first-year principal, 
continuity was possible because the principal was part of the leadership team under the former 
principal. More experienced teachers reported that as they focused on student ownership and 
responsibility over several years, student response was improving. Although the school had recently 
added new teachers in response to student growth, the newcomers were hired only after expressing 



support for Lakeside’s goals of student ownership and responsibility and willingness to enforce the 
code. The strong relationships and a sense of care between teachers and students also facilitated the 
school’s efforts to help students take more responsibility for their learning. Teachers reported that 
students were willing to be held accountable and responded to teachers' efforts to shift the locus of 
responsibility over to them because they had strong, personal relationships. Students felt that their 
teachers were willing to go the extra mile for them and they responded in kind. 

Because the Code outlined behaviors that teachers were expected to exhibit, teacher accountability 
practices facilitated the successful enactment of the Code. Student ownership and responsibility for 
learning was the focus of both student and teacher accountability, with classroom observations 
including checks for teachers’ efforts to ensure that students took active ownership of their own 
learning. Teachers, then, were held accountable for holding students accountable. 

Teachers’ willingness to be held accountable in these ways was supported by a sense of individual 
and collective teacher efficacy. While teachers described several school challenges, such as lack of 
resources, language barriers with students and parents, and low parental involvement, most teachers 
described these forces as challenges to be overcome rather than excuses for not succeeding. Teachers 
believed their efforts could affect student success, both individually within the classroom and 
collectively through school-wide practices. In particular, teachers recognized that while getting 
students to take responsibility was a major undertaking, the school was experiencing some success. 

Finally, the safe and orderly environment in the school also supported enactment of the Code. Most 
students were interested in their learning, and discipline problems were resolved efficiently and fairly. 
Teachers said the administration supported them in handling student behavior problems. These 
attributes allowed the school to focus efforts on enforcing academic expectations rather than 
behavioral expectations. 


2. Lakeside: Learning Time 

Description 

Learning Time was offered during the extended lunch hour when teachers were available to tutor 
students. Unlike other high schools in the district that had eight 45-minute periods, Lakeside had only 
seven 45-minute periods to accommodate Learning Time. Students were required to attend unless 
they had an 81 percent or above in all classes, in which case they received a pass for the six-week 
marking period. Students could choose how to spend the time and some extracurricular activities, 
such as student clubs, also met during Learning Time. Thus, a student could attend tutoring with his 
or her math teacher on Monday, tutoring with his or her science teacher on Tuesday, and a meeting of 
the drama club on Wednesday. Although some students with a Learning Time pass reported using 
that time for an extended lunch, sometimes leaving campus to eat, others occasionally sought extra 
assistance before tests or participated in clubs. 

The Lakeside Code expected students to attend tutoring for all classes in which they were struggling 
or unable to explain key ideas of lessons, and required them to show evidence that they were trying to 
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understand questions they had missed on assignments and assessments. During tutorials, students 
could work one on one with teachers, in small groups, or independently to complete missed 
assignments. The tutorials were a mix of formal and informal tutoring opportunities, and on most 
days, teachers were available to answer questions from students or help with assignments. 
Occasionally, one department would offer a more formal tutoring session if the faculty noticed many 
students having difficulty with a particular topic. All teachers were expected to be available during 
Learning Time four days a week, getting one day a week to take a full hour-long lunch. However, in 
practice, most teachers reported spending the lunch time in their classrooms for Learning Time nearly 
every day. 

Learning Time also provided opportunities for students and teachers to build personal, non-academic 
connections. Some students ate lunch, socialized with their friends, or talked to the teacher about non- 
academic concerns during this time. One participant explained that Learning Time is “not just kids 
coming for tutorials, but coming to hang out, talk to us, just be there.” 

Learning Time was in its third year of implementation in Lakeside in the 2011-2012 school year, 
though the new principal instituted several changes. Students were expected to check in with their 
homeroom teacher at the beginning of Learning Time and inform him or her where they would be 
spending the time. All teachers were expected to keep track of which students came to them for 
tutoring so homeroom teachers could check whether students actually attended. Some faculty noted 
issues with implementation of the changes and frustration with the increasing number of details. Upon 
hearing this feedback, the leadership decided to reduce the mechanisms to enforce student attendance, 
and some students reported skipping tutorials even though they did not have a Learning Time pass. 
Although students were expected to sign in for tutorial sessions and keep a tutorial log, theses data 
were not used systematically to monitor attendance or progress. Despite these inconsistencies, 
students reported in interviews and focus groups that they attended Learning Time about two to three 
times per week. This was a considerably higher rate than in the other three case study schools, 
although they also offered tutorial opportunities, sometimes even during lunch. 

How Learning Time Supported Increasing Student Ownership and Responsibility 

Learning Time was an effective strategy for increasing student ownership and responsibility because 
it provided academic support. Students and teachers throughout the school said it helped students 
meet academic expectations. By not only creating time for additional help, but by also 
institutionalizing this time in the school schedule and making it the default expectation, Learning 
Time enabled students to better engage in the challenging work set out by their teachers. By making 
Learning Time the default expectation for students, we mean that rather than operating under the 
assumption that attending tutorials was an extra, optional activity, school personnel and students 
appeared to view attending tutorials as the standard, expected behavior. This view was evident in the 
expectation for students to attend tutorials to make up work or understand concepts they did not grasp 
fully, even though Learning Time passes were awarded to students who performed well. 

The shared expectation that students would attend Learning Time to master learning goals — and the 
reinforcement of this expectation during class time — also served as a form of academic press. For 
example, teachers were observed reminding students to come to Learning Time when they could not 


answer a question in class or struggled with an assignment. 

School-wide facilitating conditions reinforced Learning Time. Shifting responsibility for learning to 
students involved creating aligned and coherent structures that supported a shared school-wide 
vision to provide multiple opportunities for students to succeed. The Lakeside Code spelled out the 
school’s standard for excellence and Learning Time and other supports provided opportunities for 
help to students falling short of the standard. Learning Time also supported use of the Assignment 
Log. Teachers were expected to post assignments in a visible area of the classroom and students were 
to keep track of both the assignment and the grade they earned. Through Assignment Logs, students 
and their teachers were expected to know their class average at any given moment, and therefore 
know whether they should be attending a tutorial during Learning Time. The responsibility for using 
this time wisely rested mainly with the students; teachers did not actively monitor where students 
were or whether they attended the correct tutorial. While teachers and students said some students did 
not use this time wisely, the general consensus was that most took advantage of this time as needed to 
complete assignments or receive extra help. Students who continued to struggle despite opportunities 
to make up work during Learning Time were referred to the Intervention Committee (see below for 
more details) to receive an individualized learning plan to monitor their attendance, Assignment Log, 
and assessments. 

Trust, stability, and care and relationships between students and teachers also supported the 
effective enactment of Learning Time. They facilitated student understanding of— and adherence to 
Learning Time requirements. While attendance at Learning Time was not perfect, students reported 
wanting to attend because they knew teachers would be available for additional help. Students told us 
they most often spent Learning Time with teachers they trusted and with whom they had strong, 
positive relationships. In turn, Learning Time reinforced these relationships as it provided additional 
opportunities for teachers and students to get to know each other. Trust among adults in the school 
and staff stability also enhanced the effectiveness of Learning Time. 

While Learning Time was not a major component of teacher accountability in Lakeside, there was 
evidence that it reinforced individual and collective teacher efficacy as it provided an avenue for 
student success and helped teachers overcome a perceived barrier — the lack of time and space in 
students’ home lives — to getting extra assistance. Teachers reported that many students used Learning 
Time to complete missed assignments and homework, tasks that work and family obligations often 
made difficult outside of school. 

Finally, the safe and orderly environment aided the enactment of Learning Time. Such a program 
could have set up a different school for disorder. Unlike regular class periods, students did not have to 
be at the same place for Learning Time each day. Coupled with the extended lunch period and fact 
that some students had Learning Time passes, the potential existed for a great deal of uncertainty 
about where students were supposed to be. However, the school remained relatively calm during this 
time 
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3. Lakeside: Intervention Committee 


Description 

The key tenets of the Lakeside Code required students to put forth effort. Students who failed to meet 
the Code’s expectations were identified for a series of interventions that culminated with the 
Intervention Committee. The first step was a conference with the teacher in whose class the student 
was struggling to determine the root cause of the poor performance and to develop a plan for 
improvement. If problems continued, the teacher reminded the student and parents about expectations 
related to the Code, and again sought to determine the reason for this failure and develop an 
improvement plan, adding steps to help the parent support the plan. If the student still failed to show 
the desired improvement, he or she was referred to the Intervention Committee. A sponsor was then 
assigned to review past interventions with the student and why they did not succeed. Ultimately, 
students who still failed to show the expected effort were assigned to in-school suspension and 
possibly referred to a hearing at the district level. The Intervention Committee was led by the dean of 
instruction and composed of the school social worker, an intervention specialist, and five teachers. 

The Committee met to discuss students as a group and all members sponsored students referred to 
them, although the caseload could vary by committee member due to other responsibilities. Sponsors 
were required to coordinate with the assistant principal who was assigned to the student. Teachers and 
Intervention Committee members were expected to work with the student to identify underlying 
causes of failure and move beyond explanations that blamed students (e.g., the student is lazy). The 
goal was to identify the underlying reasons and then develop a plan to support the student. 

The Intervention Committee was in its first year in 201 1-12. School participants estimated that 70 to 
80 percent of students referred to the Intervention Committee became successful, suggesting that 
referred students tended to start turning in homework and quiz grades improved. 

How Intervention Committee Supported Increasing Student Ownership and Responsibility 

The Intervention Committee served as a key academic support for students. The interventions were 
actively linked to academics as part of Lakeside’s push for student ownership and responsibility. 
Notably, adults in the school (starting with the administration) asserted that students’ behavior 
reflected underlying academic issues. This belief informed academic-first responses and interventions 
to problems that were not ostensibly academic. Hence, the committee sought to address the cause of 
student failure before a student became a behavioral problem. 

The Intervention Committee illustrates how Lakeside enacted the school-wide facilitating conditions 
evident in effective schools. For example, a shared vision and an explicit aligned and coherent 
school-wide practice, were intertwined. The vision focused on both outcomes and a process to 
achieve those outcomes. Students were referred to the Intervention Committee if they met two 
conditions: earning an 80 or below in any class and not meeting the expectations of the Lakeside 
Code. Thus the school set both a performance expectation and a set of behavioral expectations to help 
students meet the performance expectation. When students failed, the intervention process 
emphasized helping them understand and implement the behavioral expectations described by the 
Code, such as making up missed assignments, going to tutorials for additional help, and paying closer 


attention in class. As noted, the missions of the Code and the Intervention Committee were aligned in 
stressing the importance of practices established and supported by the school, such as Assignment 
Logs and Learning Time. 

Care and relationships between students and teachers and trust among all school actors were 
important for the smooth implementation of the Intervention Committee. Committee members had to 
know what was going on in a student’s life to mete out effective interventions. Further, the 
interventions required students to assume accountability, undergirded by more intensive monitoring 
of their academic behaviors and support. This synergistic interplay of accountability and support 
worked because of the trust and positive relationships that existed in the school. 

Teacher accountability was also tied to the intervention process because when administrators met 
with teachers about student failure, they asked whether the expected steps had been followed: had 
teachers tried interventions and attempted to identify the root cause of failure before referring 
students to the Intervention Committee? Administrators reported holding teachers with high student 
failure rates accountable for following the process. 

The process is not unique to Lakeside. What was somewhat unusual was the focus on academics 
rather than behavioral disruptions, made possible by the safe and orderly environment and absence 
of significant student behavioral problems. 


4. Riverview: Increasing Enrollment in Advanced Courses 

Description 

Administrators, faculty, and students at Riverview described access to Advanced Placement and 
honors courses for all students and low-income minority students in particular as a priority. 
Administrators and teachers described this as an effort to increase inclusivity at the school — a key 
lever to provide greater learning opportunities for a broad spectrum of the student population. The 
faculty was reported to be committed to making “honors students” out of regular achievers. 

Similarly, a major goal was to make sure students got the highest level of education possible, with a 
two-pronged school- wide strategy that 1) increased the number of AP/honors-level courses while 
protecting their quality and 2) encouraged more students to enroll in these courses. While the district 
policy allowed students to choose AP/honors courses, teachers, assistant principals, and counselors at 
Riverview used proactive strategies to identify and encourage more regular-level students to enroll in 
challenging courses. These strategies included conversations between faculty and students (e.g. 
teachers encouraging students to enroll in an AP class) as well as “behind the scenes” or “back 
channel” discussions between faculty and counselors and assistant principals to identify students 
likely to succeed in honors or AP courses. 

Riverview had a long reputation among district high schools for its success in preparing students for 
college. Access to advanced courses seemed much higher than at the other case study schools, and 
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was accompanied by higher rates of enrollment and, importantly, passing scores on AP exams. 
Riverside also had structured the curriculum to provide options to appeal to varied interests. For 
example, students interested in English did not have to stop with Advanced Placement English 
Literature; the school also offered Linguistics as an additional option. Similarly, students talented in 
math could take differential equations. Only one other high school in the district had a higher 
percentage of students taking advanced courses. 

How Increasing Enrollment in Advanced Courses Supported Increasing Student Ownership 
and Responsibility 

Encouraging students to enroll in accelerated courses was one way in which Riverview instituted high 
academic press and helped to increase students’ ownership of and responsibility for their learning. 
Administrators, teachers, and students described high expectations for students in AP/honors level 
courses, and many students, in turn, were described as having high expectations for themselves. Their 
success hinged on taking ownership for their own learning. For example, the more rigorous in-class 
and out-of-class work expected of AP and honors courses demanded organization, engagement, the 
timely completion of assignments, preparation for assessments, and participation in challenging class 
discussions. An additional benefit was that students in higher-level courses were exposed to habits of 
mind and practices that had value beyond the academic curriculum. 

Academic support was a crucial ingredient for helping students succeed in higher-level courses. One 
manifestation was the school-wide expectation that all teachers offer tutoring after school. 
Additionally, counselors and teachers described meetings in which they encouraged struggling 
students, and if applicable their parents, to remain in advanced courses. 

In general, a sense of trust existed in the school and was often described as enabling relationship 
building and a sense of caring between teachers and students. Teacher-student trust supported 
efforts to move students into more advanced courses by increasing students’ willingness to work hard. 

The school’s stability, evidenced by the lack of transience among teachers and guidance counselors, 
allowed for ongoing support for moving students into more advanced coursework. The principal, 
although only in the second year as school leader, was committed to pushing this practice as a means 
to close gaps between the perceived “two schools within a school.” 

The school had teacher accountability mechanisms, such as department-level walk-through 
evaluations and a formal teacher evaluation program provided by the district. However, we found no 
evidence that teachers were held accountable for ensuring student movement into more advanced 
courses. As described previously, this practice was more a “professional commitment”. 

Examples of individual and collective teacher efficacy emerged in discussions of advanced course 
curricula. Evidence indicated that teachers of advanced courses often increased their expectations and 
provided challenges to match students’ abilities. These teachers also had more influence over the 
curriculum and could enrich lessons with extra material to engage and challenge students. Counselors 


collectively believed more students could excel in higher-level courses, and leveraged their 
familiarity with students’ profiles and their relationships with them to guide them into advanced 
courses when working on schedules. 


5. Valley: A VID Program 

Description 

All middle schools and high schools in the district have AVID (Advancement via Individual 
Determination) programs a national program designed to promote college readiness and increase 
school-wide performance and learning. Valley, in particular, used the program to help increase 
students’ ownership and responsibility for the own learning. 

At Valley, AVID was considered an honors program, focusing on college readiness for the sizeable 
majority of students who would be the first in their family to attend college. Approximately 9 percent 
of students participated, with many continuing participation from middle school. While teachers 
could identify and recommend additional students for the program, our interviews suggested that it 
was nearly at capacity and additional recruitment was not a priority. AVID functioned as an elective 
course, teaching organizational skills (e.g., using binders, time management with a planner, Cornell 
Notes); writing skills; exploration of college benefits and support in the application process (e.g., 
careers that require a postsecondary education, guidance on filling out applications, getting fees 
waived for the SAT/ACT, and completing student aid and scholarship applications); academic 
tutoring; and participation in community service. AVID students also were expected to take an 
increasing load of honors or AP courses as they progressed through their sophomore, junior, and 
senior years. 

How the AVID Program Supported Increasing Student Ownership and Responsibility 

The AVID program focused on holding students accountable for meeting the program's expectations. 
Because the program was voluntary teachers emphasized to students that participation involved 
additional commitment and responsibility. Further, the national AVID program exemplifies a balance 
between academic press and academic support. AVID students in Valley were expected to exhibit 
positive social behavior, take rigorous courses, and adopt organizational and study skills that would 
prepare them for college. They also became personally invested in the goal of college attendance. 
While college attendance and success were clear outcome goals, strategies to increase student 
ownership toward those goals involved specific activities, such as keeping materials in binders, 
organizing class notes and summaries, using Cornell notes, taking an increasing number of honors 
classes, and applying for college admissions, scholarships, and financial aid. While AVID students 
were held accountable for these organizational processes, as well as their behavior and performance 
in their classes, students reported feeling supported by their AVID teacher. The AVID teacher 
reported that the school provided supplies such as binders when possible and expected students to 
take care of them. Replacements were available for defective supplies but the idea was to instill 
responsibility. 

Expectations for AVID students were academic and behavioral, with the program addressing both 
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socio-emotional and academic needs. The AVID teacher reported visiting students in other classes 
when teachers reported misbehavior as a technique to hold students accountable and provide supports 
to help them meet expectations in all classes. 

AVID also capitalized on relationships and a sense of care between students and their AVID teachers. 
Students described how the program supported their preparations for college, with teachers helping 
them conduct the college search, complete the application process, and find scholarship money. 
Students also reported that the main AVID teacher was accessible and open to conversations about 
many topics, indicating that the AVID teacher and tutors are involved with students, respect them, 
and do not give up on them. 

The strength of Valley’s AVID program was a result of the stability of leadership within the AVID 
program as well as a culture of trust between students and AVID leadership. The AVID leadership 
had run the program for several years, and students who remained with the program over multiple 
years forged deeper connections and stronger relationships with the staff. 

We highlight AVID in Valley, which was identified as a lower value-added school, because it 
appeared to be a practice that helped increase students’ ownership and responsibility for their 
academic success. This case demonstrates how some effective practices were observed in lower 
value-added schools, even if they only existed in pockets. The size and voluntary nature of the AVID 
program (e.g., students and parents signed a contract and could be “exited” if they did not meet 
expectations), would make effective scaling to the whole school a significant challenge. 


6. Valley: The Challenge 

Description 

Teachers in Valley used mandatory tutoring or in-school suspension to hold students accountable for 
completing assignments and improving their test performance. The approach was not universally 
successful, however. Some teachers continued to struggle with getting students to turn in homework, 
while others only assigned a minimal amount. Teachers had the option of assigning behavioral 
sanctions if students did not show up for mandatory tutoring, but one core content area department 
attempted to shift responsibility for completing work or getting assistance onto students by turning 
mandatory tutoring into “the Valley Challenge.” The “challenge” was for students to compete on a 
quiz, knowing they must come for extra help if they earned less than 8 1 percent, thus giving them 
more responsibility for their own achievement. The administration is tracking the results of this more 
voluntary program and will compare it to the existing mandatory tutoring program. 

How the Valley Challenge Supported Increasing Student Ownership and Responsibility 

The Valley Challenge illustrates how the academic department in Valley worked to support students 
in challenging themselves to exceed the minimum requirements. By using competition instead of 
coercion, the teachers pushed the ownership of academic success back onto students, reinforcing the 
message that they were in charge of academic outcomes. While this approach was just beginning at 


our time in the school (making the evidence of its success limited), the department hoped that 
motivating students to score well in the competition would increase attendance at tutorials, increase 
engagement during class, and increase individual students’ beliefs that they could master challenging 
objectives. 

The unity of this department in attempting a new approach to tutorials demonstrated a shared vision 
among staff and a sense of collective teacher efficacy. After reflecting on current tutorial practices, 
teachers in this department recognized the limitation of their current approach, brainstormed possible 
solutions, and collectively attempted the Valley Challenge. 

The Challenge was also supported by trust between leadership and faculty. In breaking away from 
the school-wide tutorial plan, this department needed the trust of school leaders to allow it to 
experiment and improve student outcomes. Without this trust, the department may never have 
launched the Challenge. 

Finally, The Challenge was facilitated by positive relationships and a sense of care. The 
Challenge’s goal was to avoid the perception of punishing students if they did not attend tutoring 
(e.g., by having to spend lunch with an assistant principal) and to encourage teachers to develop 
relationships and trust with students so they wanted to learn. This departmental team tried to improve 
student learning by focusing first on changing standard practices that had a negative connotation into 
ones about which students would feel more positive. 


7. Mountainside: Junior Reserve Officer Training Corps (JROTC) 

Description 

Nationwide, the Junior Reserve Officer Training Corps (JROTC) prepares students at over 3,000 high 
schools for leadership roles, making them aware of their rights, responsibilities, and privileges as 
American citizens. It encourages graduation from high school. The curriculum teaches students self- 
discipline, confidence, and leadership skills. The program is conducted by instructors who are retired 
Navy, Army, Air Force, Marine Corps, and Coast Guard officers and enlisted personnel. 

JROTC relies on a curriculum called the Leadership Education Training (LET) program that is 
generated by the national ROTC administration. This curriculum emphasizes citizenship, character 
and leadership development, and community service. Cadets learn about the elements of leadership, 
drill instruction and ceremonies, military customs, uniform inspections, physical fitness training, 
marksmanship, and military history. Community service activities, drill competition, field trips, 
marksmanship training, and other extracurricular activities augment classroom instruction. The 
military provides uniforms, textbooks, training aids, and a substantial portion of instructors’ salaries. 
The commander at Mountainside attributed part of the program’ s success to its authentic curriculum 
that taught life skills such as financial planning among many others. He reported that keeping the 
curriculum in tune with students’ lived experiences kept them engaged. 
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How JROTC Supported Increasing Student Ownership and Responsibility 

As evidenced by shadowing observations of students’ class days, JROTC at Mountainside contributed 
to increased cognitive and behavioral engagement through student ownership and responsibility. 
JROTC participants reported that the program supported academic achievement by focusing acutely 
on the literacy of incoming freshmen. This provided integrated support and accountability for 
students. Thus, academic press was intrinsic to the program and supported by the Leadership 
Education Training (LET) curriculum and culture of responsibility and high expectations. The 
program at Mountainside had high rates of graduation (98%), postsecondary enrollment (50%), and 
military attachment (25%). 

Further, the program illustrated the enactment of school-wide facilitating conditions in the service of 
encouraging students to take responsibility for their learning, albeit for only a portion of the student 
body. For example, it fostered care and relationships between students and teachers. The program 
helped students feel connected to the school and provided a positive sense of family cohesion. One 
participant reported that the program helped establish relationships with students, adding that s/he 
“serves as a mentor for a number of kids in my program.” Care and relationships between students 
and teachers were also evident in the commendation and celebration of student academic achievement 
through JROTC promotion ceremonies. While parental involvement was reportedly low in the school 
as a whole, the JROTC program successfully engaged parents through activities such as promotion 
ceremonies and field trips. 

The JROTC program included various types of systemic performance accountability. The program 
set clear expectations for student performance. In addition, adults in the JROTC program were held 
accountable for performance. One JROTC instructor reported that s/he was evaluated by the principal, 
as well as ROTC cadet command (an external entity), and that s/he faced significant potential 
consequences from federal inspections. Lastly, students were held accountable. JROTC students and 
instructors reported that cadets were held to high standards within the program, including standards of 
dress, conduct, and academic success. 

Further, Mountainside has had a JROTC program for over five decades, a sign of the program’s 
stability. In the 2011-12 school year, it served an estimated 16 percent of students, many of whom 
were recruited from the JROTC program in the sole feeder middle school. Alumni of the program 
have gone on to universities and careers in the armed forces. The commander and the instructors 
shared a common vision for the program, partially by virtue of its affiliation with the national 
program. The goal of the national program is “to prepare cadets to meet the challenges and demands 
in the 21st Century.” 


Conclusion and Next Steps. 


Increasing student ownership and responsibility for their academic success holds promise for raising 
student achievement in high schools, based both on the case study research presented here and prior 
research that links the elements of student ownership and responsibility with student outcomes." 
Further, teaching students the academic behaviors and engagement strategies to take responsibility for 
their learning (e.g., going to class, doing homework, organizing materials, studying) can lead to more 
positive outcomes in college. 100 Drawing on intensive case studies of four high schools in Fort Worth, 
Texas, this report underscores the importance of student ownership and responsibility. While most of 
the findings on student ownership and responsibility came from one higher value-added school, 
evidence existed in the second higher value-added school of other levers to increase academic press 
and rigor. Furthermore, we also found pockets of practices in the lower value-added schools that 
contributed to student ownership and responsibility. While increasing student ownership and 
responsibility may not be easy, requiring that teachers and other adults in the school provide careful 
scaffolding, it is noteworthy that Lakeside - one of the highest poverty schools in the district - 
achieved success despite constrained resources. 

Further, the findings from these case studies suggest that the higher value-added schools effectively 
raised student achievement for traditionally underserved students (i.e., economically disadvantaged, 
racial minority, and ELL) by developing, integrating, and aligning school-wide goals and processes 
that cut across the essential components of effective schools, serving as the glue to hold them 
together. We refer to these processes as school-wide facilitating conditions. To be clear, it was not the 
school’s focus on increasing student ownership and responsibility alone that explained achievement 
results. Rather, it was the fact that this focus was integrated and sustained by the school-wide 
facilitating conditions and built on a set of foundational elements that supported school-wide efforts. 

The next stage of the Center’s work involves bringing district leaders, school leaders, and teachers 
together to collaborate in the design and implementation of an innovation to increase students’ 
ownership and responsibility for their own academic success in other FWISD high schools. In this 
way, the central findings from this report will define a design challenge to guide a collaborative 
design process. A District Innovation Design Team (DIDT) will develop an innovation based on the 
research findings presented in this report, the broader research literature on effective practices, and a 
needs assessment on what aspects of student ownership and responsibility are currently in place in 
other high schools. School Innovation Design Teams (SIDTs) will pilot, adapt, and implement the 
innovation in three low value-added schools. As part of this process, schools themselves will also be 
studying and evaluating the impact with an eye to scaling up. The researchers in the Center will then 
study and evaluate this implementation, examine its impact, and assess the district’s ability to support 
and scale up the interventions to additional high schools. 
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Appendix A: Data and Methods 


The data in this report come from a comparative case study of four high schools in FWISD during the 
2011-2012 school year. The school selection process is first described, followed by a description of the 
data, and then the analytic methods. 

School Selection 

The schools for this comparative case study were selected based on school-level value-added measures. 101 
In short, two had relatively higher value-added results and two had relatively lower value-added results. 
Specifically, school-level value-added models were estimated for high schools in FWISD based on 
assessments in math, reading/ELA, and science for grades 9-11, the high school grades tested by the state. 
The value-added results were calculated for both the 2010-201 1 school year and the three-year average 
for the 2008-09, 2009-10, and 2010-1 1 school years. Aggregating results from three years of data and 
comparing the three-year average value-added results with the 2010-201 1 school year results allowed for 
cross-validation of consistency. Additionally, value-added results were computed for each subject area for 
selected student subgroups: free- and reduced-price lunch (FRL, a measure of poverty), limited English 
proficiency (LEP), black, and Hispanic. 

The FWISD value-added model measures the average test-score improvement among a school’s students, 
while controlling for prior achievement and a large number of student characteristics, including gender, 
race/ethnicity, FRL, special education, at-risk status, gifted and talented, LEP, and whether the student 
was retained in grade or skipped a grade between the pre-test and post-test years. The model can be 
defined by the following four equations: 
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where i denotes a student, k denotes a school, g denotes grade, t denotes time, denotes an 
alternate-subject test, and 

y tikg = true ability for student i in grade g of school k in year t. 

Ytikg = TAKS test score for student i in grade g of school k in year t. 

v (t— i) = measurement error for student i in grade g of school k in year t. 

x ti kg = vector of demographic variables on student i in grade g of school k in year t. 

a kg = grade g of school k’s constant effect value-added. 


Equation (1) states that a student’s posttest score is equal to the sum of a constant ( Q, the same-subject 

pretest and its slope parameter ( ^ v ^ j an alternate-subject pretest and its slope parameter 

( , s ;t alt the vector of student characteristics with its slope parameter , t n and an error 

A g yCt-DikC^-DJ \. X tikgPg) 

term ( .. ). In other words, a student’s posttest score can be measured by the two pretests and the 

*tikg 

student’s characteristics (in this model, gender, race, English language learner status, FRL status, special 
education status, at risk status, gifted and talented status, and grade retention between the pre- and post- 
test years). A constant is also included to account for any shift in the student’s achievement between the 
pretest and posttest, and the remaining discrepancy is accounted for by an error term that captures any 
error that may result from student achievement measurements and inherent biases of the data. Hereafter, 
v and v are used interchangeably to indicate the posttest. Similarly, y and y are 

Y likg Y tikg F Y Oikg Y {t-l)ik(g-l) 

used interchangeably to refer to the pretest, time subscripts from all variables are dropped. 

Equations (2), (3), and (4) state that a student’s knowledge cannot be perfectly measured by each test in 

the model — in this case, the posttest and the two pretests. In other words, a student’s true knowledge of a 

subject (as measured by alt \ is only partially observed by the subject tests 

y-mg*ywkg>y®ikgj 

f V v v ait , respectively). The fact that there are three equations — one for the posttest and two 
V/lifce' Y 0 ikg> Y 0 ikg 

for the pretests — state that each assessment has its own measurement error, which is determined by 
individual test characteristics, such as the difficulty of test questions, the number of tested topics, and the 
number of question items. The model adjusts for measurement error in lagged test scores using standard 
errors of measurement (SEMs), using approaches described in Wayne Fuller’s Measurement Error- 
Models (Wiley, 1986). It is important to adjust for pre-test measurement error to avoid underestimating 
the pre-test coefficient because such underestimation would bias estimates of the demographic and 

value-added coefficients p and 


a- 


Since 


, , v , and ait are not observable, equations (2), (3), and (4) can be substituted into 

0 ikg >1 ikg >o ikg 

n (1), which yields an equation defined in terms of measured student achievement: 
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The new error term 


in equation (6) now includes both the original error component ( , ) and the 

tikg tikg 


measurement 


error components (, , ; ). Moreover, the error term in the model 
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allowed to have a variance that differs across subjects, grades, and years but is identically independently 
distributed (i.d.d.) across students within each combination of subject, grade, and year. This makes the 
model a “T2” model that uses two consecutive years of assessment data to produce value-added results 
for a given growth year. As a result, regression (6) can be run separately by subject, grade, and year. 
Equation (6) can also be more simply understood as a regression of a student’s 201 1 TAKS score 
(posttest) on the student’s 2010 TAKS score (pretest), a vector of demographic variables, and a vector of 
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dummy variables that indicates the school the student was enrolled in during the 2010-201 1 school year. 


Finally, the grade-level value-added results are then aggregated to the school level by calculating the 
weighted average by the number of students in each grade. The resulting value-added of each FWISD 
school is then ranked for each core, tested subject (i.e., reading, math, and science). One overall value- 
added estimate is then calculated by weighting the averages of the three value-added estimates of each 
subject by the number of students. Similar value-added estimates are also estimated for student subgroups 
based on race/ethnicity, LEP status, and economic disadvantage. The described analyses yield multiple 
value-added results for each school. Each school in the tested grades in the 2010-11 school year has a 
single -year overall value-added measure for 2010-1 1; a multiple-year overall value-added that averages 
results for 2008-09 through 2010-11; single-year value-added measures for the 2010-1 1 school year 
specific to student subgroups (black, Flispanic, ELL, special education, and FRL); and multiple-year 
value-added measures that cover the same subgroups for all three school years. 

Of the 14 high schools in FWISD, one was too new to have enough data to provide a value-added 
estimate. Two were excluded from the case study selection because their special admission requirements 
made it hard to compare them to other schools without those admission processes. One high school was 
excluded from school selection due to district request. This left 10 candidates for selection. The value- 
added estimates and ranks were then considered alongside school demographic characteristics, state 
accountability rankings, graduation rates, and dropout rates when selecting schools. The goal was to 
identify the two highest and two lowest value-added schools while balancing graduation and dropout rates 
and ensuring similar demographic characteristics between the higher and lower value-added schools. 
Finally, the school selection process was conducted in such a way that the research team visiting the 
schools did not know prior to the first visit whether the school was selected as a higher or lower value- 
added school. 

Selecting schools on the basis of value-added indicators for multiple subgroups in multiple subjects is not 
a simple task. School-level value-added measures also vary even within the same school for different 
subgroups of students or for different subjects. This study includes two schools that have relatively high 
value-added measures for most student subgroups and subjects. We refer to them as Lakeside and 
Riverview. Lakeside is ranked first or second in value-added measures for most subgroups of students, 
but fell near the bottom for LEP students. Riverview was one of the most highly ranked schools in value- 
added for the student body as a while as well as LEP, and black students, but was in the middle of the 
district for FRL students and near the bottom for Hispanic students. Both higher value-added (HVA) 
schools have graduation rates above 85%. The two schools with relatively low value-added measures are 
Valley and Mountainside. Valley has value-added measures that rank it near the bottom of the district 
when combining all subjects and all students, but its value-added measures in mathematics were above 
the district average. Valley also was near the top of the district for value-added for LEP students. 
Mountainside had the lowest value-added measure when combining all students and all subjects and for 
FRL and black students, but its value-added measures were closer to the middle of the distinct for 
Hispanic students. Both lower value-added (LVA) schools had graduation rates below 80%. 



Data Collected 


Data were collected in Fort Worth in three different time periods in the 2011-12 school year: Wave 1 in 
late November - early December 2011; Wave 2 in mid-late February 2012; and Wave 3 in late April - 
early May 2012. The last wave took place after students took STAAR/TAKS exams and before Advanced 
Placement exams. Data were drawn from focus groups (with students, teachers, student activity leaders, 
district parent liaisons); interviews with principals, assistant principals, guidance counselors, support 
personnel, teachers, students, and district personnel; observations in English, mathematics, and science 
classrooms; shadowing of students during their regular school day; and the collection of school and 
district artifacts. In addition to this fieldwork, we also obtained administrative, disciplinary, and course- 
taking data from the district. We also collaborated with the district to obtain survey data from teachers, 
students, parents and principals. Table A-l below shows the amount of data collected by wave and data 
type. 


Table A-l. Types and Amounts of Data Collected by Wave 


Data type 

Wave 1 

Wave 2 

Wave 3 

Total 

Interviews 

80 

82 

56 

218 

School Administrators 

4 

15 

2 

21 

Teachers 

72 

0 

1 

73 

Deans of Instruction 

0 

3 

2 

5 

Department Heads/Lead Content Teachers 

0 

21 

0 

21 

Guidance Counselors 

0 

15 

0 

15 

Support Personnel 

0 

24 

11 

35 

Students 

0 

0 

37 

37 

District Administrators 

4 

4 

3 

11 

Focus groups 

22 

0 

14 

36 

Students 

11 

0 

1 

12 

Teachers 

11 

0 

0 

11 

Student Activity Leaders 

0 

0 

12 

12 

District Parent Liaison 

0 

0 

1 

1 

Observations 

138 

128 

51 

317 

Classroom Periods 

138 

128 

8 

274 

Students Shadowed 

0 

0 

37 

37 

Faculty/School Administrative Team Meetings 

0 

0 

6 

6 

Total 




571 


Note: Teachers and other school personnel may have participated in more than one type of data 
collection. For example, some individuals may have been interviewed both as a teacher and a department 
head or lead content teacher (LCT). Similarly, a teacher may have participated in a general teacher focus 
group in Wave 1 and then the student activity leader focus group in Wave 3 due to his or her role as an 
athletic coach. 
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Interviews and focus groups 

Data collection primarily focused on 9 th - and 10 th -grade students and teachers in English, mathematics, 
and science, although we balanced this focus with other data from key staff and a cross-section of the 
school (e.g., teacher focus groups spanned all grades and subject areas) to gain a comprehensive 
understanding of our schools. Table A-2 shows the types and amount of data collected from each case 
study school. 


Table A-2. Types and Amounts of Data Collected by School 



LVA Schools 

HVA Schools 

Data type 

Mountainside 

Valley 

Lakeside 

Riverview 

Interviews 

51 

50 

54 

48 

School Administrators 

4 

5 

5 

7 

Teachers 

18 

19 

18 

18 

Deans of Instruction 

2 

2 

1 

0 

Department Heads/Lead Content Teachers 

6 

6 

3 

6 

Guidance Counselors 

2 

4 

3 

2 

Support Personnel 

9 

5 

16 

5 

Students 

10 

9 

8 

10 

Focus groups 

9 

8 

9 

9 

Students 

3 

3 

3 

3 

Teachers 

3 

3 

3 

3 

Student Activity Leaders 

3 

2 

3 

3 

Obsen’ations 

76 

81 

87 

78 

Classroom Periods 

66 

72 

73 

68 

Students Shadowed 

10 

9 

8 

10 

Faculty/School Administrative Team 
Meetings 

0 

0 

6 

0 


All principals, assistant principals, guidance counselors, and deans of instruction (if applicable) were 
interviewed. The principal (or other key school leaders in the case of two schools where the principal was 
not available in Wave 3) were interviewed twice. Six teachers in each of the mathematics, ELA, and 
science departments were interviewed (and observed) in each school, for a total of 1 8 teachers per school. 
These teachers were chosen because they taught classes designed for 9 th - and 10 th -grade students. In three 
of the schools, we observed all the 9 th -and 10 th -grade teachers, while in the fourth school, because of its 
size, we sampled all 10 th -grade teachers and additional -grade teachers so that six teachers in each core 
subject were included. All department heads and lead content teachers (LCT) in the three targeted 
subjects were interviewed (in one school, these roles were held by the same person). Other support 
personnel were sampled in two different ways. First, individuals with specific roles in the school, such as 
special education, LEP coordinators, and testing coordinators were interviewed. Second, we used a 


snowball sampling technique to interview school personnel who were identified by other participants as 
serving in key roles. These roles included, for example, stay-in-school coordinators, coordinators for 
programs working with students considered at risk, and parent liaisons. 

We conducted three types of focus groups. First, teachers who were not sampled for individual interviews 
were invited to participate in focus groups. These groups were designed to get a wider representation of 
teachers, including those teaching subjects other than mathematics, English, and science as well as 
teachers in those subjects for students in grades 1 1 and 12. About eight teachers participated in each focus 
group. 

Second, we conducted focus groups with students who had been selected on the basis of grade level and 
course selection patterns. We focused on students in grades 10 to 12 because they were more familiar 
with their schools, although some 9 th -graders did participate in some focus groups. Student focus groups 
were organized to include one for students taking primarily “advanced” courses, one for students taking 
primarily “general” courses, and one for students enrolled primarily in “remedial” or “basic” classes. 
Students were selected based upon on the convenience of their schedules with the goal of having a cross 
section in each focus group that broadly represented the demographics of students in that course selection 
pattern. 

Because our initial data analysis highlighted the importance of extracurricular activities in engaging 
students, we also conducted focus groups in Wave 3 with teachers and other adults who supervise these 
activities to learn more about how they were manifested in the school. Student activity leaders were 
sampled to ensure representation from activities in five areas: sports, community service, academic focus 
(i.e., poetry club, academic competition groups), social clubs (i.e., manga club, prom committee), and 
programs/structures provided by the school to engage students (i.e., AVID, after school programs, 
JROTC). Further, we attempted to include activities that served both small and large numbers of students. 

District administrators were selected to focus on areas of responsibility that included the following: direct 
responsibility for our case study schools (i.e., secondary directors); curriculum and instruction units; high- 
school reform; multicultural and English for Speakers of Other Languages (ESOL) program services; and 
the deputy superintendent of educational programs and student support. In addition to interviews with 
district administrators, we conducted a focus group with four parent liaisons assigned to multiple schools 
by the district. 

We observed some of the administrative, departmental, and new teacher meetings in one school. A 
structured observation log was used for these observations. Finally, we also interviewed 13 administrators 
from the district central office to get their perspectives on the case study schools in particular and 
activities or elements that facilitate high school effectiveness in general. The types of district 
administrators interviewed included assistant superintendents, those who oversaw the case study schools, 
personnel who oversaw programs for selected student populations, and curriculum directors for literacy, 
math, and science. Appendix F presents our findings on district perceptions of high school effectiveness. 
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Classroom observations (CLASS-S) 

We observed and videotaped a total of 274 class periods of English Language Arts (ELA), math, and 
science. The same teachers who participated in the interviews were also observed. Classroom 
observations focused on 9 th - and 10 th -grade classrooms because these grades were the basis for most of 
the assessment data in the value-added measures. Four class periods per teacher were videotaped and 
coded by trained observers. In most cases, two class periods were observed in Wave 1 and two were 
observed in Wave 2. A few observations that were missed due to teacher absences or scheduling 
difficulties were made up in Wave 3. 

We used an observational tool called the CLASS-S (Pianta.et al., 201 1) to assess teacher-student 
interactions. We observed and coded the following domains and dimensions using the CLASS-S 
framework: Emotional Support (positive climate, negative climate, teacher sensitivity, regard for 
adolescent behavior), Classroom Organization (behavior management, productivity, instructional learning 
formats), and Instructional Support in the classroom (content understanding, analysis and problem 
solving, quality of feedback, and instructional dialogue), and Student Engagement. 

The CLASS-S protocol is designed to code 20-minute segments. Because the classes were either 45 or 90 
minutes long depending on the school schedule, each class period had two or three observation segments. 
Thus the 274 classroom videos represent a total of 603 20-minute observation segments for CLASS-S 
coding. Inter-rater reliability was calculated on approximately 20% of the observation segments which 
were double coded (i.e., two raters coded the same segment independently, then reconciled discrepancies 
and came to consensus on the scores). Of the 127 segments that were double-coded, overall exact point 
reliability was 43 percent, and one-point reliability was 90 percent. 

See Appendix B for additional information on the classroom observations. 

Shadowing 

During Wave 3 of our data collection in spring 2012, we conducted 37 student shadowing observations in 
which a member of the research team accompanied a student through his or her school day. After students 
were shadowed, they were interviewed using a semi-structured interview guide designed to elicit their 
perspectives on things observed during the shadowing experience and ask them about their experience at 
the school. Students in each school were selected based on the level of courses they typically took — four 
to five who took mostly advanced (Advanced Placement, pre-AP, or honors) classes and four to five who 
took mostly “on-level” or “regular” classes. We tried to ensure that they sample reflected the broader 
student body demographics in terms of gender, race/ethnicity, and free/reduced lunch status. Students 
who had participated in focus groups were not eligible to participate in the shadowing observations or 
interviews. 

One researcher followed each student throughout his or her entire school day (with the exception of 
restroom visits and lunch) using a structured observation log. Every five minutes, the researcher recorded 
data in an electronic shadowing log, noting the class period, precise time, course subject, course track, 
location, the teacher’s expectations of the student (what the student was supposed to be doing), the 
academic nature of that task (i.e., related to content or not), nature of student engagement in that task 
(active engagement, passive engagement, not engaged), and the behavior the student in engaging in if he 


or she was off-task, and with whom the student was interacting. The types of activities that teachers 
expected students to engage in included: whole class discussion, direct instruction, pair or group work, 
individual work, an interactive or student-led activity, taking a test or quiz, transitioning between 
activities, other academic activities (includes watching or giving a presentation, general studying, 
watching a film or video, or academically-oriented talk with the teacher), non-academic activities (such as 
socially-oriented talk with the teacher, handing out report cards, saying the pledge, school 
announcements), other (for non-core subjects, this includes doing ROTC drills and playing sports in gym 
class), and nothing (i.e., there is nothing the student is supposed to be doing at that moment). 

See Appendix C for more information on the student shadowing observations. 

Surveys and administrative data 

We collaborated with the district to obtain survey data from students, teachers, parents, and principals to 
further understand the processes, programs, and practices that might explain school effectiveness. The 
surveys were administered by the district across all schools, not just in the four case study high schools. 
This report uses data from the teacher and student surveys only. To maintain confidentiality of principals, 
we do not report principal survey data in this report. Due to low response rates on the parent survey (an 
average of 7% at the high school level), we do not report parent survey results. 

The student survey was administered to students in grades 9, 10, and 1 1 in November 201 1 . A total of 
10,827 high school students completed surveys, representing approximately 70% of enrolled 9 th to 1 1 th - 
grade students. Response rates of those students in the four case study schools ranged from 72% to 91%. 
Demographic differences between survey respondents and the total enrolled population across the high 
schools were slight, with females and Hispanics being slightly overrepresented in the survey sample. 
Factor analyses were then performed separately on principal components of each survey construct. The 
student survey measured the following constructs: academic engagement, personalization, parent press 
toward academic achievement, peer support for academic achievement, student sense of belonging, 
student study habits, how well the school prepared students for successful futures, school climate, disaster 
preparation, the degree to which students felt expectations for academic press, the degree to which 
students felt challenged academically, the degree to which students showed responsibility through class 
participation, the degree to which the school culture fostered responsible behavior,, school safety, 
bullying, and parent connections. See Appendix D for more information on the student survey. 

The teacher survey was administered in January 2012. Across the district high schools 577 teachers 
completed the survey, for an overall response rate of 44%. Response rates within the four schools varied 
considerably, ranging from 30% to over 60% of teachers. Factor analyses of principal components were 
performed separately on each of the proposed survey constructs. Constructs included: bullying, data use, 
efficacy, instructional program coherence, personalization-social, school leader instructional support, 
teacher-principal trust, teacher-teacher trust, supporting quality instruction, systemic performance 
accountability, supportive and shared leadership, expectations for postsecondary education, 
personalization-school action, teacher accountability, teacher outreach to parents, teacher-parent trust, and 
provision of time to collaborate. More information on the teacher survey can be found in Appendix E. 

Finally, the district provided a comprehensive set of administrative data from the past 10 years, or since 
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the start of the systematic collection of that data if it was more recent. The administrative data sources 
included student discipline, attendance, Advanced Placement (AP) and other advanced course-taking, AP 
exam scores, teacher characteristics, student characteristics, and graduation and dropouts. 

Analyses of administrative data are presented as school-level means or percentages, with significance 
level calculated through OLS linear regression on school dummy variables, with comparison group as the 
remaining teachers and students (n schools=13). 


Data Coding and Analysis 

For the interviews, focus groups, and observations conducted in our four case study schools we employed 
a multi-stage approach to analyze researchers’ field notes. Field notes were kept in two forms: participant 
interaction forms (Miles & Huberman, 1994), which were completed by researchers within 24 hours of 
conducting an interview or moderating a focus group; and school-level analysis forms, which were 
completed by the members of each school’s research team together during the first week-long visit. These 
served as inputs for generating a cross-school comparison matrix that compared schools across the 
essential components that guided our work (see below for information on the essential components). 
These three types of documents provided the basis for engaging in the iterative process of refining our 
instruments and planning our next field visit. Our analyses were guided by our core research questions: 
What are the distinguishing characteristics between higher and lower value-added schools? How did 
these differences develop and how are they orchestrated and supported? 

In summer 2012, we embarked on an in-depth cross-case analysis. A team of 19 people systematically 
coded the interview and focus group data using NVivo, a qualitative analysis software program (QSR 
International, 2011). We used the analytic technique of explanation building (Yin, 2009) to 
understand how and why the essential components developed (or did not develop) in our schools. This 
work was iterative and involved continuously refining claims about the school as additional evidence was 
examined. We also engaged in directed content analysis (Hseih & Shannon, 2005) to code items for both 
our essential components and themes that cut across the essential components. The following guiding 
principles framed our analytic work: 

• Focusing on answering our core research questions; 

• Discerning findings that lead us to a “design challenge” for Fort Worth; 

• Establishing a process that is rigorous and systematic and allows for tracking claims back to the 
data/evidence and allows us to return to the data and evidence for each finding; and 

• Maintaining the essential components as an analytic frame. 

To meet these principles, the work was spread among four teams, one for each school. Each team had 
four to six members. All but one team member had first-hand experience collecting the fieldwork data in 
that school. School-based teams were responsible for coding and analyzing all data collected about that 
school and writing a comprehensive case report. Using an emergent, inductive approach to coding, every 
member of a school team read through seven to eight key transcripts, selected in advance to include the 
fieldwork notes and comparison tables created after each visit, the principal transcripts, and data from 



selected teacher and student focus groups. The school team then met to develop an emergent coding 
framework that was grounded in the data (Glasser and Strauss, 2007). In addition we used an a priori 
coding scheme of our essential components and cross-cutting facilitating conditions and themes (i.e., 
goals, trust, locus of control, structures that support or inhibit goals, rigor and academic press, student 
culture of learning, and student responsibility). The general approach was to look at each school as a 
system. That is, the school is the unit of analysis. Our analysis centered on understanding each school in- 
depth, while maintaining a focus on the essential components within each school, as well as additional 
facilitating conditions that emerged. School case teams met weekly for about four hours. In between 
meetings, team members coded interview and focus group transcripts. 

In addition, we held cross-case comparison meetings involving all four school teams every other week for 
approximately three hours. These meetings had two goals: to ensure that definitions were being applied 
consistently and reliably across schools in the coding process, and to flag emerging findings about each 
school to begin making comparisons across schools. 

Once all interview and focus group data were coded, school-level teams developed a narrative of each 
essential component. Coders strove to provide a thorough, well-supported set of claims about the drivers 
and/or inhibitors of essential components, as well as the practices and policies through which these were 
enacted. 

Developing case reports 

To write the school-level case reports, each school team was asked to address the following questions: 

• What is the context in which this school operates and how does the context affect school 
processes? 

• What are the key characteristics of the school that enable (or impede) development of the 
essential components? 

• What features/processes/practices in the school serve as drivers (or inhibitors) of school outcomes 
and essential components? 

• Describe the nature of each essential component in the school. How did it develop? How is it 
enacted and supported? 

Using emergent, grounded theory allowed for a more iterative analytic process that enhanced the 
interrogation of the claims being made, adding to the internal validity (Patton, 2002). It also permitted 
more explicit attention to the themes/facilitating conditions that cut across components and explicit 
testing of emerging hypotheses and triangulation among different types of data, which added to the 
validity of process (Patton, 2002). 

With the school-level case reports as our base, we held an intensive set of cross-case meetings to look 
systematically across the cases to build cross-case explanations. This process was used to note the 
presence or absence of differences between HVA and LVA schools. Each team member read all four case 
reports in their entirety to ensure that we thoroughly understood of each case. Next, multiple people were 
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assigned to conduct cross-school analyses of particular components, cross-cutting themes, key findings, 
and the quantitative data (surveys and district administrative data). The purpose was to identify the 
differences between the higher and lower value-added schools and explain what contributed to these 
differences in the context of FWISD. 



Appendix B: Quality of Classroom Instruction Report 


Data Collection 

To assess the quality of classroom instruction across our four case study schools, we targeted lOth-grade 
English/language arts, mathematics, and science classes. Eighteen teachers, with six from each of the 
three major content areas, from each school were selected for observation. These teachers were sampled 
because they taught classes designed for 9 th - and 10 th -grade students. Due to the moderate size of three of 
the schools, we observed all the 9 th - and 10 th -grade teachers. In the fourth school, we sampled all 10 th - 
grade teachers and additional 9 th -grade teachers to ensure that six teachers in each core subject were 
included. Data collection focused on 9 th - and 10 th -grade classrooms because those grades produced most 
of the assessment data used to calculate the value-added measures. 

Researchers videotaped participating teachers’ instruction during the same class period on two 
consecutive days during the Wave 1 visits (late November to early December) and Wave 2 visits (mid- 
late February). Eight make-up observations occurred during the Wave 3 visit (late April to early May) for 
observations that were missed due to teacher absences or scheduling difficulties. Thus, most teachers 
were observed four times. The total number of class periods observed was 66 in Mountainside, 72 in 
Valley, 73 in Lakeside, and 68 in Riverview. Videographers logged the sequence of activities that 
occurred in the classroom, collected student assignments, and recorded student demographics. 

The CLASS-S framework is designed for raters to watch either live or video classroom instruction for at 
least 20-minute segments while taking notes on the CLASS-S indicators. Then they take 10 to 15 minutes 
to review their manual and assign scores to each domain. Regular class period observations of 40 to 50 
minutes were evenly divided into two segments. For Valley, which uses block scheduling with about 90- 
minute class periods, each observation was divided evenly into three segments. Due to teacher absences 
and attrition from the study, between 1 and 12 segments were coded for each of the 72 teachers and the 
274 classroom videos represent a total of 603 observation segments for CLASS-S coding. Table B-l 
shows the number of observational segments in each school, by subject and track. 


Table B-l. Number of Observational Segments by School 



LVA Schools 

HVA Schools 

Combined 

Subject 

Mountainside 

Valley 

Lakeside 

Riverview 

Mathematics 

37 

70 

44 

43 

194 

Advanced classes 

19 

24 

22 

27 

92 

Non-advanced classes 

18 

46 

22 

16 

102 

English/Language arts 

44 

70 

48 

40 

202 

Advanced classes 

22 

23 

28 

24 

97 

Non-advanced classes 

22 

47 

20 

16 

105 

Science 

41 

72 

48 

46 

207 

Advanced classes 

11 

24 

24 

27 

86 

Non-advanced classes 

30 

48 

24 

19 

121 

Total 

122 

212 

140 

129 

603 


Reaching for Rigor 


Research Report I October 2013 99 



As research on tracking in high schools suggests that higher track classes tend to have higher quality 
instruction than lower track classes, we wanted to assess whether this was occurring in our cases study 
schools, as well as whether higher value-added schools “compressed” the instructional quality between 
their higher and lower track classes more than lower value-added schools. Observed class periods were 
selected to ensure a number of honors, pre-AP, AP, and other advanced classes were observed in each 
school. However, we did not observe the same teacher teaching in both types of classes. 

These classroom observations were coded using the Classroom Assessment Scoring System - Secondary 
(CLASS-S), developed by Robert Pianta, Bridget Hamre, Nancy Hayes, Susan Mintz, and Karen LaParo. 
The CLASS was originally designed to measure preschool and early elementary teachers’ instructional 
practices. CLASS assesses the quality of teachers’ social and instructional interactions with students as 
well as the intentionality and productivity evident in classroom settings. The focus of the CLASS is on 
what teachers do with the materials they have and on their interactions with students, rather than on a 
particular curriculum, lesson format, or the physical setup of the classroom. The original CLASS was 
developed based on an extensive literature review as well as on scales used in large classroom observation 
studies in the National Institute of Child Health and Human Development (NICHD) Study of Early Child 
Care and the National Center for Early Development and Learning (NCEDL) Multi-State Pre-K Study. 
Dimensions were derived from the review of constructs assessed in instructions for conducting classroom 
observation in child care and elementary school research, focus groups, and extensive piloting. 
Dimensions are based on development theory and research suggesting that interactions between students 
and adults are the primary mechanism of student development and learning. The greatest distinction 
between the CLASS-S and the other versions of the CLASS lies in the Instructional Support domain 
where Content Understanding and Analysis and Problem Solving have the place of Concept Development 
and Language Modeling has been eliminated (Pianta et al., 2007). Table B-2 shows the major domains of 
CLASS-S, the dimensions within these domains, and the indicators used to represent the dimensions. 



Table B-2. Overview of 2011 CLASS-S Dimensions, Domains, and Indicators 


Domains 

Dimensions 

Indicators 

Emotional 

Support 

Positive Climate 

• Relationships 

• Positive affect 

• Positive communications 

• Respect 

Negative Climate 

• Negative affect 

• Punitive control 

• Disrespect 

Teacher Sensitivity 

• Awareness 

• Responsiveness to academic and social/emotional 
needs and cues 

• Effectiveness in addressing problems 

• Student comfort 

Regard for Adolescent 
Perspective 

• Flexibility and adolescent focus 

• Connections to current life 

• Support for student autonomy and leadership 

• Meaningful peer interactions 

Classroom 

Organization 

Behavior Management 

• Clear expectations 

• Proactive 

• Effective redirection of misbehavior 

• Student behavior 

Productivity 

• Maximizing learning time 

• Routines 

• Transitions 

• Preparation 

Instructional Learning 
Formats 

• Learning targets/organization 

• Variety of modalities, strategies, and materials 

• Active facilitation 

• Effective engagement 

Instructional 

Support 

Content Understanding 

• Depth of understanding 

• Communication of concepts and procedures 

• Background knowledge and misconceptions 

• Transmission of content knowledge and procedures 

• Opportunity for practice of procedures and skills 

Analysis and Problem 
Solving 

• Inquiry and analysis 

• Opportunities for novel application 

• Metacognition 

Quality of Feedback 

• Feedback loops 

• Scaffolding 

• Building on student responses 

• Encouragement and affirmation 
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Instructional Dialogue 

• Cumulative content-driven exchanges 

• Distributed talk 

• Facilitation strategies 

Student 

Engagement 


• Active engagement 


Coders rated each dimension as low (1,2), mid (3, 4, 5), and high (6, 7). While the CLASS manual 
provides general scoring guidelines (see Table B-3), it notes that “observers should view the dimensions 
as holistic descriptions of classrooms that fall in the low, mid, or high range.” 


Table B-3. CLASS-S General Scoring Guidelines 


Low 


Mid 


High 

1 

2 

3 

4 

5 

6 

7 

The low 

The low 

The mid- 

The mid- 

The mid- 

The high 

The high 

range 

range 

range 

range 

range 

range 

rang 

description 

description 

description 

description 

description 

description 

description 

fits the 

mostly fits 

mostly fits 

mostly fits 

mostly fits 

mostly fits 

fits the 

classroom/ 

the 

the 

the 

the 

the 

classroom/ 

teacher very 

classroom/ 

classroom/ 

classroom/ 

classroom/ 

classroom/ 

teacher very 

well. All, or 

teacher but 

teacher, but 

teacher very 

teacher, but 

teacher, but 

well. All, or 

almost all, 

there are one 

there are one 

well. All, or 

there are one 

there are one 

almost all, 

relevant 

or two 

or two 

almost all, 

or two 

or two 

relevant 

indicators in 

indicators 

indicators in 

relevant 

indicators in 

indicators in 

indicators in 

the low 

that are in 

the low 

indicators in 

the high 

the mid- 

the high 

range are 
present. 

the mid- 
range. 

range. 

the mid- 
range are 
present. 

range. 

range. 

range are 
present. 


Note. From CLASS-Secondary Manual (Pianta et al., 201 1) 


The CLASS has been used by teacher preparation programs and for teacher performance assessment, 
professional development, program monitoring, and research and evaluation. Research has found that 
students in classrooms with higher CLASS scores make greater academic and social gains than those in 
classrooms with lower CLASS scores, though most of this comes from studies conducted at the preschool 
and elementary level. Positive correlations have been found between algebra end-of-course exams 
(EOCs) and the CLASS-S domains. The highest correlation was Classroom Organization and the lowest 
was with Emotional Support (Bell, Gitomer, McCaffrey, Flamre, & Pianta, 2011). 

Researchers at the Educational Testing Service (ETS), the University of Virginia, and the RAND 
Corporation are currently exploring best practices in implementing the CLASS-S. The study’s goals are 
(1) continue to develop and refine a theoretically driven and well-researched CLASS for secondary 
settings, (2) determine the validity and reliability of this system for secondary classrooms, and (3) 





understand the relationships among classroom characteristics, processes and outcomes. The CLASS-S is 
also currently being used in the Measures of Effective Teaching (MET) project funded by the Bill & 
Melinda Gates Foundation, which is working to develop and test multiple measures of teaching 
effectiveness. Some initial findings from the MET project found that the CLASS and CLASS-S 
(considered together) were positively associated with student achievement gains in both math and ELA. 
Further, this study found that rating reliability is improved if at least four lessons are observed and coded 
by multiple observers, which is consistent with the practices used in our study. 

Coding and Reliability 

Observation rating was completed after each wave of data collection. The rating team consisted of nine 
raters who completed a live or online training course and were certified by Teachstone, a company that 
oversees the training of the CLASS observation tools. Before the team individually rated videos, all raters 
watched two segments of different teachers and came together to discuss any discrepancies and decide on 
a uniform set of ratings. Raters were randomly assigned to segments so that the same rater did not rate 
more than one segment of a particular teacher from a single visit. 

Video observation allowed for ongoing inter-rater reliability checks. Acceptable reliability on the 
CLASS-S occurs when raters are within one point of each other 80 percent of the time (Pianta et al., 

201 1). With 12 possible dimension scores for each segment rated, raters should be within one point of 
each other on 10 of the 12 dimensions. To assess ongoing inter-rater reliability, 20 percent of the videos 
were randomly selected for double rating. Rather than averaging the ratings of the videos that were 
double rated, consensus rating was used. Consensus rating was not only a means of assessing inter-rater 
reliability, but the frequent consensus sessions also served as an ongoing means of ensuring a shared 
understanding of the measures. 102 Raters were randomly assigned to code segments, and then raters were 
randomly assigned to act as second coders for 20 percent of the segments. The two raters met to discuss 
rating disagreements and arrive at a consensus. Consensus codes were entered as the final codes. If at any 
point that the two raters did not meet the 80 percent-one-point standard across all segments rated, a third 
person rated the segment as well. Third raters were needed in 15 instances. Of the 127 segments that were 
double -rated, overall exact point reliability was 43 percent, and one-point reliability was 90 percent. 

Table B-4 shows inter-rater reliability information for CLASS-S coding. 


Table B-4: Percentage of inter-rater reliability by wave of data collection 


Wave 

Segments 

Coded 

Segments 
Double coded 

Percent Exact 
Agreement 

Percent 
agreement 
within 1 point 

Percent agreement 
within 2 points (across 
all possible codes) 

1 

302 

62 

41% 

88% 

98% 

2 

284 

61 

44 

91 

98 

3 

17 

4 

50 

98 

100 

Total 

603 

127 

43 

90 

98 
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Results 

We first present the findings in Table B-5. 
Table B-5. CLASS-S Scores by School 



LVA Schools 

HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

Combined 

Emotional Support 

Positive Climate 

4 4q*h=* 

4.91 

4.93 

4.94 

4.82 

Negative Climate 

6.24 


6.53** 

6.63*** 

6.36 

Teacher Sensitivity 

4.34*** 

4.69 

4.96** 

4.96* 

4.74 

Regard for Adolescent 

2 90 *** 

3.23 

3.49* 

3.43 

3.26 

Perspectives 
Organizational Support 

Behavior Management 

5.05*** 

5.25* 

5 92*** 

5.61 

5.44 

Productivity 

5.07** 

5.15** 

5.68*** 

5.56* 

5.34 

Instructional Learning 

4.33* 

4.40* 

4.76* 

4 79 ** 

4.55 

Formats 

Instructional Support 

Content Understanding 

4.49 

4.37* 

4.60 

4.68 

4.52 

Analysis and Problem 

2.42** 

2.58 

3.03*** 

2.78 

2.70 

Solving 

Quality of Feedback 

3.85 

3.75* 

4.26*** 

3.85 

3.91 

Instructional Dialogue 

3.18 

3.23 

3.57** 

3.22 

3.30 

Student Engagement 

4.39** 

4.58 

4.83* 

4.88* 

4.67 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on mean 
comparisons tests between each case study school’s mean rating compared to the mean from the other 
schools combined. 


Emotional Support 

The Emotional Support domain measures these characteristics: positive climate, negative climate, teacher 
sensitivity, and regard for adolescent perspective. In general, higher value-added schools had higher 
ratings for the Emotional Support dimensions than the lower value-added schools, with these differences 
statistically significant in all areas except positive climate. Additionally, advanced classes had higher 
average Emotional Support ratings, and these differences were statistically significant in all dimensions 
except teacher sensitivity. 

Positive climate reflects the emotional connections and relationships among teachers and students, and the 
warmth, respect, and enjoyment communicated by verbal and non-verbal interactions. Indicators of this 
dimension included positive relationships, positive affect, and positive communications, and respect. 
Lakeside, Riverview, and Valley had similar average positive climate scores, while Mountainside had the 


lowest ratings. The differences between the high and lower value-added schools were not statistically 
significant for positive climate. Advanced classes had higher average positive climate than non-advanced 
classes, and this difference was statistically significant. 

Negative climate encompasses the overall level of negativity among teachers and students in the observed 
class. This variable has been reverse coded, so a higher score reflect less negative climate. Indicators of 
negative climate include negative effect, punitive control, and disrespect. On average, higher value-added 
schools had better negative climate ratings than the lower value-added schools, and this difference was 
statistically significant. Non-advanced track classes tended to have worse negative climate ratings than 
the advanced classes across all four schools, and this difference was statistically significant. 

Teacher sensitivity reflects the teacher’s responsiveness to the academic and social/emotional needs and 
developmental levels of individual students and the entire class, and the way these factors impact 
students’ classroom experiences. Indicators of teacher sensitivity included awareness, responsiveness to 
academic and social/emotional needs and cues, effectiveness in addressing problems, and student 
comfort. On average, higher value-added schools had better teacher sensitivity scores than lower value- 
added schools, and this difference was statistically significant. Teacher sensitivity scores tended to be 
higher in advanced classes than they were in non-advanced classes, though these differences were not 
statistically significant. 

Regard for adolescent perspectives focuses on the extent to which the teacher is able to meet and 
capitalized on the social and developmental needs and goals of adolescents by providing opportunities for 
student autonomy and leadership. It also considers the extent to which student ideas and opinions are 
valued and content is made useful and relevant to adolescents. Indicators of regard for adolescent 
perspectives include support for student autonomy and leadership, meaningful peer interactions, 
flexibility, and connections to current life, student ideas and opinions. On average, ratings on regard for 
adolescent perspective tended to be low, with teachers scoring in the lower end of the mid-section of the 
rating scale. The higher value-added schools had higher ratings on average for this measure, and this 
difference was statistically significant. Advanced courses tended to have higher scores than non-advanced 
courses on this measure across schools, with Riverview and Valley having the widest gaps. The 
difference in ratings was significant across tracks. 

Organizational Support 

The Organizational Support domain includes behavior management, productivity, and instructional 
learning formats. In general, higher value-added schools showed greater Organizational Support than 
lower value-added schools, although Valley had higher ratings than Mountainside. Additionally, 
advanced courses tended to receive higher Organizational Support scores than non-advanced classes, with 
statistically significant differences across tracks in all dimensions of organizational support. The gap 
between tracks tended to be smallest in Mountainside. 

Behavior management focuses on the teacher’s use of effective methods to encourage desirable behavior 
and prevent and redirect misbehavior. Indicators of positive behavior management include clear 
expectations, proactive responses by the teacher in situations where behavior issues are likely to arise, and 
effective redirection of misbehavior. On average, higher value-added schools had better behavior 
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management ratings than lower value-added schools, with the difference statistically significant, holding 
constant track, grade, and subject. Behavior management scores tended to be higher in advanced classes 
than in non-advanced classes, and the difference was statistically significant. 

Productivity considers how well the teacher manages time and routines so that instructional time is 
maximized. Indicators of productivity include maximizing learning time, evidence of routines, and 
frequent transitions. Average productivity ratings were higher in Lakeside than in Riverview, the higher 
value-added schools, and the difference was statistically significant. On average, advanced classes had 
higher productivity ratings than non-advanced classes, with the largest gaps at Lakeside and Valley and 
the narrowest gap at Mountainside. The difference among advanced and non-advanced tracks was also 
statistically significant. 

The instructional learning formats measure focuses on ways in which the teacher maximizes student 
engagement through clear presentation of material, active facilitation, and the provision of interesting and 
engaging lessons and materials. Indicators include evidence of learning targets/organization, active 
facilitation effective engagement, and variety in modalities, strategies, and materials. Higher average 
instructional learning formats ratings were observed in higher value-added schools, and this difference 
was statistically significant. As with other Organizational Support measures, advanced courses tended to 
have higher instructional learning formats ratings, on average, than non-advanced classes, with these 
differences being statistically significant. 

Instructional Support 

The Instructional Support domain consists of content understanding, analysis and problem solving, and 
quality of feedback. In general, higher value-added schools had higher Instructional Support scores than 
lower value-added schools. These differences were generally statistically significant, but this was likely 
driven by the low scores of Mountainside. Advanced courses tended to receive higher Instructional 
Support scores than non-advanced classes. 

Content understanding refers to both the depth of lesson content and the approaches used to help students 
comprehend the framework, key ideas, and procedures. Indicators of content understanding include 
demonstration by the teacher of deep understanding, effective communication of concepts and 
procedures, demonstration of background knowledge and misconceptions, and effective transmission of 
content knowledge and procedures. On average, higher value-added schools had slightly better average 
scores for content understanding than lower value-added schools, though the difference was not 
statistically significant. Across all four schools, advanced classes had higher content understanding 
scores, with the largest gap at Valley and the smallest at Lakeside. The difference across tracks was 
statistically significant. 

Analysis and problem solving assesses the degree to which the teacher facilitates students’ use of higher 
level thinking skills, such as analysis, problem solving, reasoning, and creating through the application of 
knowledge and skills. Indicators included opportunities for higher level thinking, problem solving, and 
metacognition. On average, analysis and problem solving received the lowest scores of all the CLASS-S 
dimensions. The higher value-added schools had higher average analysis and problem solving scores than 
did the lower value-added schools, and this difference was statistically significant. Here too tracking 


mattered, although the narrowest gap was in a lower value-added school (Mountainside), while the widest 
was at the other lower value-added school (Valley). The difference in scores across tracks was statistically 
significant. 

Quality of feedback looks at the way the teacher’s feedback expands and extends learning and 
understanding and encourages student participation. In secondary classrooms, this dimension 
acknowledges that peers may also provide feedback. Indicators include evidence of teachers providing 
feedback loops, prompting thought processes, scaffolding instruction, and providing information, 
encouragement and affirmation. In general, the two higher value-added schools had higher average 
quality of feedback scores than did the lower value-added schools, with Lakeside having the highest 
average quality of feedback score. This difference was marginally statistically significant. Although most 
schools had a tracking a gap, differences in the sizes of gaps were not statistically significant. 

Instructional dialogue captures the puiposeful use of dialogue (questioning and discussion) by teachers to 
facilitate students’ understanding of content. The indicators for instructional dialogue include cumulative 
content-driven exchanges, distributed talk, and facilitation strategies. As with quality of feedback, 
Lakeside had the highest average instructional dialogue score, and Mountainside had the lowest. The 
differences between the high and lower value-added schools was not statistically significant. On average, 
advanced classes had higher instructional dialogue scores than non-advanced classes, and this difference 
was marginally statistically significant. The widest gap between tracks was at Valley. 

CLASS-S also assesses Student Engagement. This dimension is not part of a larger domain. It is intended 
to measure the degree to which students in the class are focused and participating in learning activities. 
Indicators include active and sustained engagement in the lesson. The higher value-added schools had 
higher average student engagement scores than the lower value-added schools. The difference was 
marginally statistically significant. Advanced track classes were significantly associated with higher 
student engagement scores than lower track classes. The gap between advanced and non-advanced classes 
was widest at Lakeside. 

Figure B-l shows the distribution of ratings across the dimensions and Tables B-7 through B-10 display 
the average CLASS-S scores by track in each school. 
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Figure B-l. Distribution of CLASS-S Scores by Dimension 


Table B-8. Mean CLASS-S Scores by Track for Moun tainside 


Dimensions 

Advanced 

Non-advanced 

Emotional Support 

Positive Climate 

4.62 

4.24 

Negative Climate 

6.35 

6.16 

Teacher Sensitivity 

4.54 

4.20 

Regard for Adolescent Perspectives 

2.87 

2.93 

Organizational Support 

Behavior Management 

5.19 

4.94 

Productivity 

5.10 

5.06 

Instructional Learning Formats 

4.42 

4.26 

Instructional Support 

Content Understanding 

4.54 

4.46 

Analysis and Problem Solving 

2.40 

2.43 

Quality of Feedback 

3.92 

3.80 

Instructional Dialogue 

3.08 

3.25 

Student Engagement** 

4.69 

4.17 


Note: The difference between advanced and non-advanced courses is statistically significant on measures 
as indicated by * for p<.05, ** for p<.01, and *** for p<.001. 


Table B-7. Mean CLASS-S Scores by Track for Valley 


Dimensions 

Advanced 

Non-advanced 

Emotional Support 

Positive Climate* 

5.14 

4.79 

Negative Climate*** 

6.51 

5.99 

Teacher Sensitivity 

4.86 

4.60 

Regard for Adolescent Perspectives*** 

3.62 

3.03 

Organizational Support 

Behavior Management *** 

5.75 

5.00 

Productivity** 

5.48 

4.98 

Instructional Learning Formats** 

4.72 

4.24 

Instructional Support 

Content Understanding*** 

4.83 

4.14 

Analysis and Problem Solving*** 

3.07 

2.34 

Quality of Feedback 

3.97 

3.63 

Instructional Dialogue*** 

3.79 

2.95 

Student Engagement*** 

4.99 

4.38 


Note: The difference between advanced and non-advanced courses is statistically significant 
measures as indicated by * for p<.05, ** forpc.Ol, and *** for pc.OOl. 

Table B-9. Mean CLASS-S Scores by Track for Riverview 

Dimensions 

Advanced Non-advanced 

Emotioned Support 



Positive Climate*** 

5.21 

4.53 

Negative Climate** 

6.78 

6.39 

Teacher Sensitivity 

5.04 

4.84 

Regard for Adolescent 

3.65 

3.08 

Perspectives** 



Organizational Support 



Behavior Management*** 

5.94 

5.12 

Productivity 

5.68 

5.37 

Instructional Learning Formats 

4.87 

4.67 

Instructional Support 



Content Understanding 

4.78 

4.52 

Analysis and Problem Solving 

2.91 

2.59 

Quality of Feedback 

4.04 

3.57 

Instructional Dialogue 

3.37 

3.00 

Student Engagement** 

5.14 

4.49 


Note: The difference between advanced and non-advanced courses is statistically significant on 
measures as indicated by * for p<.05, ** forpc.Ol, and *** for pc.OOl. 
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Table B-10. Mean CLASS-S Scores by Track for Lakeside 


Dimensions 

Advanced 

Non-advanced 

Emotioned Support 

Positive Climate* 

5.14 

4.70 

Negative Climate 

6.61 

6.44 

Teacher Sensitivity 

5.03 

4.89 

Regard for Adolescent Perspectives 

3.50 

3.47 

Organizational Support 

Behavior Management *** 

6.28 

5.48 

Productivity*** 

5.91 

5.41 

Instructional Learning Formats*** 

5.01 

4.48 

Instructional Support 

Content Understanding 

4.72 

4.47 

Analysis and Problem Solving** 

3.31 

2.71 

Quality of Feedback 

4.24 

4.29 

Instructional Dialogue 

3.54 

3.61 

Student Engagement*** 

5.26 

4.35 


Note: The difference between advanced and non-advanced courses is statistically significant on measures 
as indicated by * for p<.05, ** for p<.01, and *** for p<.001. 



Appendix C: Fort Worth Student Shadowing Report 


Data Collection and Analysis 

In Wave 3 of data collection (late April to early May 2012), we shadowed 10 th -grade students as they 
went about their normal daily activities. The goal was to understand how students experienced the school. 
Every five minutes, the researcher recorded data about where the student was, what the student was 
expected to be doing, the student’ s level of engagement, and with whom the student was interacting. 

There was also space for additional notes about that five-minute segment. Table C-l shows the number of 
students and total observational segments per school. Some observational segments were excluded from 
analysis. For example, observations were excluded when a Lakeside student missed three class periods 
due to a dentist appointment since that time segment revealed no information about typical student 
experiences at school. Some students were observed during lunch, homeroom classes, and/or tutorial 
periods, although access during these periods varied within and among schools. Because these parts of the 
day were not uniformly observed, they were excluded from the analysis. Finally, observations that 
occurred during the transition between classes were also excluded since this analysis is intended to focus 
on activities during class. 


Table C-l. Number of Students and Five Minute Observational Segments for Student Shadowing 



LVA Schools 

HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

Combined 

Number of students 

10 

9 

8 

10 

37 

Total observation segments 

737 

678 

602 

777 

2794 

Excluded segments 

Dentist 

0 

0 

35 

0 

35 

Transition 

50 

24 

45 

55 

174 

Homeroom/T utorial/Lunch 

12 

24 

43 

70 

149 

Number observation segments in all- 

675 

630 

479 

652 

2436 

subject analysis 

Number observation segments in core 

387 

301 

316 

436 

1440 

subject analysis 


The shadowing log contained the following information for each observational segment: the class period, 
clock time, subject area, academic track, location, the teacher’s expectation of the student (what the 
student was supposed to be doing), the academic nature of that task (i.e., related to content or not), nature 
of student engagement in that task (active engagement, passive engagement, not engaged), what off-task 
behavior the student showed, and who the student was interacting with. The teacher’s expectation of the 
student included many options for the researcher to choose (and another category). This analysis 
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combines some seldom-used categories. The type of activity in which teachers expected students to 
engage included: whole class discussion, direct instruction, pair or group work, individual work, taking a 
test or quiz, transitioning between activities, other academic activities (such as watching or giving a 
presentation, general studying, watching a film or video, or academically-oriented talk with the teacher), 
non-academic activities (such as socially-oriented talk with the teacher, handing out report cards, saying 
the pledge, school announcements), other (for non-core subjects, this included doing ROTC drills and 
playing sports in gym class), and nothing (i.e., there was nothing the student was supposed to be doing at 
that moment). In addition to noting the type of activity, observers also assessed the nature of student 
engagement in this activity according to three categories: not engaged, passively engaged, and actively 
engaged. Active student engagement included asking questions, responding to questions, volunteering 
information, sharing ideas, or manipulating materials. Students who were actively engaged were on task 
and focused on their class-related goals. Passive engagement included behaviors such as listening but not 
responding to questions, not asking questions, and being involved but appearing disinterested in the 
assigned task. Students were considered not engaged if they were unresponsive, disinterested, distracted, 
or involved in off-task behaviors. 

Chi-square tests were used to examine differences between schools first, and then between the higher and 
lower value-added schools (combining the two schools of each type). There were three main analyses to 
explore differences between schools. First, observation segments in all courses were considered and are 
presented below. Second, only observations in the four core academic courses (English, math, science, 
and social studies) were considered. These findings were reported in the body of the report. Third, 
differences between tracks within schools were explored. For the track differences, only core academic 
subjects were considered, primarily because a track was not indicated for many non-core classes. This 
analysis used two track groupings: advanced (e.g., AP, pre-AP, honors), and non-advanced (which 
included missing track information). Inclusion, sheltered language, and remedial classes were considered 
together with on-level classes in the non-advanced category because there were too few observations in to 
consider these tracks separately. Note that students may have been observed in both advanced and non- 
advanced classes. For example, a student may have been enrolled in a pre-AP English class but a regular 
level mathematics class. 

Results for All Subjects 

Table C-2 presents the results for the shadowing analyses for all subjects, including electives. Across all 
schools, students were observed about 95% of the time to have an expected task on which they should be 
working. Student engagement varied by school, with students in Valley observed more often to be 
actively engaged in the activity and students in Eakeside observed most often to be not engaged. 

Across all the schools, students were most likely to be expected to engage in individual work (25%), 
followed by listening to direct instruction (18%). There were differences between schools in the activities 
in which teachers were expecting students to be engaged during the shadowing observations. Students in 
Mountainside were observed more often in activities that involved whole class discussion and less often 
in activities that involved direct instruction. Students in the other lower value-added, Valley, were more 
likely to be observed in interactive or student-led activities or ones that were hard to classify, and less 
likely to be expected to engage a transitional or non-academic activity. Students in Riverview were less 



likely to be assigned individual work. 

Students across the schools were most likely to be observed interacting with no one (47%), followed by 
interacting with at least one student (25%). Students in Lakeside and Valley were even more likely than 
their peers in other schools to be observed interacting with no one during class time. Students in 
Riverview were more likely to be observed interacting with just their teacher. 


Table C-2-Shadowing Results for All Subjects 



LVA Schools 

HVA Schools 



Mountainside 

Valley 

Lakeside 

Riverview 

Combined 

Engagement 

Actively engaged 

27.7% 

35.3%*** 

25.5%** 

24.9%** 

28.5 

Passively engaged 

53.5 

49 g*** 

50.9** 

59.5** 

53.7 

Not engaged 

18.8 

249*** 

23.6** 

15.7** 

17.8 

Type of Expected Activity 

Whole class discussion 

2 g 2 *** 

6.5*** 

7.9* 

9.7 

10.8 

Direct instruction 

1 1 9*** 

19.5 

16.7 

23 0*** 

17.8 

Group/pair work 

13.6 

15.9 

14.4 

15.3 

14.8 

Individual work 

28.4* 

25.1 

28.6* 

29 o*** 

25.1 

Test/Other academic 

13.2 

12.9 

14.8 

17.6* 

14.6 

Transition/Non- 

11.6 

7.6** 

24 g*** 

9.2 

10.5 

academic/No task 

Other/unclear 

3 3*** 

12.5*** 

2 9 *** 

6.1 

6.4 

With whom interacting 

Teacher 

12.3 

g 3 *** 

g 3 *** 

24.5*** 

13.8 

Student(s) 

27.4* 

24.4 

21.5 

23.9 

24.6 

Teacher and student(s) 

10 . 1 * 

7.6 

6.1 

7.4 

7.9 

No one 

40 2*** 

51.6** 

56.2*** 

41.1** 

46.5 

Other/unclear 

10 . 1 *** 

8.1 

7.9 

3.1*** 

7.3 


*School estimate has a statistically significant difference from all other schools combined at p<.05 
**School estimate has a statistically significant difference from all other schools combined at p<.01 


***School estimate has a statistically significant difference from all other schools combined at p<.001 
Note: These figures refer to percent of five-minute observational segments. For example, in 28.5% of 
observational segments for all schools combined, the student was observed to be actively engaged. 


Track Differences Within Schools 

Table C-3 compares shadowing observations between advanced and non-advanced courses within the 
same school. There are no statistically significant differences between tracks in whether students had an 
expected task or whether the task was academic in nature in any of the schools. Interestingly, there were 
no differences in student engagement across tracks in the two lower value-added schools, although there 
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were differences in the higher value-added tracks. In Lakeside, students in non-advanced classes were 
more likely to be actively engaged than their peers in advanced classes. Conversely, students in non- 
advanced classes in Riverview were more likely to be not engaged while students in advanced classes 
were more likely to be passively engaged. 

Mountainside did not have any statistically significant differences between tracks in the types of activities 
teachers expected of students. There were no differences between tracks in any of the schools on whether 
students were observed doing group or pair work or doing a non-academic or transitional activity. 

Students in both higher value-added schools were more likely to be expected to listen to direct instruction 
when in advanced classes compared to non-advanced classes. There were other differences between 
tracks in Lakeside, Riverview, and Valley, although there was not a consistent pattern among the schools 
or among the higher value-added schools. 

There also were no statistically significant differences between tracks on the question of which students 
were observed interacting with, with the exception of students in non-advanced classes interacting with 
people who are hard to classify. A number of differences emerged between tracks in Lakeside, Riverview, 
and Valley, although there was neither a consistent pattern among the four case study schools nor among 
the higher value-added schools. 



Table C-3: Student shadowing data by track and school, in the four core subjects only 



Mountainside 

Valley 

Lakeside 

Riverview 


Advanced 

Non- 

advanced 

Advanced 

Non- 

advanced 

Advanced 

Non- 

advanced 

Advanced 

Non- 

advanced 

Engagement 

Actively engaged 

22 . 8 % 

23.3% 

35.6% 

27.6% 

10 . 6 %** 

24.7%** 

20.7% 

16.6% 

Passively engaged 

61.6 

55.2 

52.9 

55.7 

60.6 

53.2 

69.6* 

58.7* 

Not engaged 

15.5 

21.5 

11.5 

16.8 

28.9 

22.0 

9 g*** 

24 7*** 

Type of Expected Activity 

Whole class discussion 

22.4% 

16.4% 

26.4%*** 

3.3%*** 

2 . 6 %** 

13.4%** 

18.7%*** 

6.4%*** 

Direct instruction 

13.3 

17.0 

9.9* 

19.5* 

37 4*** 

14_9*** 

39 o*** 

20 . 1 *** 

Group/pair work 

18.1 

11.9 

20.9** 

19.1** 

17.4 

10.0 

13.9 

20.9 

Individual work 

30.0 

26.4 

12 i*** 

32 4*** 

13.0*** 

39 3*** 

11.2 

17.3 

Test/Other academic 

10.5 

15.8 

6 . 6 * 

16.2* 

18.3*** 

5.5*** 

12.3** 

22.9** 

Transition/Non- 

5.7 

7.9 

7.7 

7.1 

11.3 

14.9 

4.8* 

11.7* 

academic/No task 

Other/unclear 

0.0 

1.7 

4.4 

2.4 

0.0 

1.0 

0.0 

0.8 

With whom interacting 

Teacher 

15.2% 

10.7% 

14.3% 

11 . 0 % 

4.4% * 

12.9%* 

42.8%*** 

15.7%*** 

Student(s) 

29.1 

23.2 

30.8 

21.4 

26.1*** 

1 1 4*** 

15.5*** 

29 3*** 

Teacher and student(s) 

10.5 

10.7 

11.0* 

3.8* 

4.4 

4.5 

10.2** 

3.2** 

No one 

45.2 

48.0 

42.9** 

59.1** 

62.6 

62.7 

26.7 

49.8 

Other/unclear 

q o*** 

7 2 *** 

1.1 

4.8 

2 . 6 * 

8.5* 

4 

2 o*** 

N 

210 

177 

91 

210 

115 

201 

187 

249 


*School has a statistically significant difference between tracks at p<.05 
** School has a statistically significant difference between tracks combined at p<.01 
*** School has a statistically significant difference between tracks at p<.001 
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Appendix D: Student Survey Data 


Data Description 

We collaborated with the district to obtain student survey data. Specifically, we coordinated the 
survey items we identified as important to this study (guided by our research questions and 
framework of essential components). Most of these survey items were incorporated into the 
district-administered survey to each stakeholder. The surveys were administered across all schools 
in the district, not just in the four case study high schools that were the focus of this study. 

The student survey was administered to students in grades 9, 10, and 1 1 in November 2011. Note 
that while 12 th -grade students were not intended to be included in this student survey (the district 
has a separate survey for 12 th -graders), some 12 th -grade students did complete the survey. Two of 
the case study schools had relatively high percentages of 12 th -graders completing the survey. A 
total of 10,425 students in grades 9 to 1 1 completed surveys, representing approximately 70% of 
enrolled students. Response rates for students in those grades in the four case study schools ranged 
from 72% to 91%. Demographic differences between survey respondents and the total enrolled 
population across the high schools were slight, with females and Hispanics being slightly 
overrepresented in the survey sample relative to total enrollment. See Table D-l for demographic 
characteristics of the student survey sample. 

We conducted a missing data analysis to investigate patterns of skipping items. Across all items, 
the amount of skipping per item ranged from 2% to 12% of the respondents. Several patterns were 
evident in the missing data analysis. Items at the end of the survey were more likely to be skipped 
than items at the beginning. Male students, black and Asian students, economically disadvantaged 
students, and English language learners were more likely to skip items. White students were less 
likely to skip items. Though fewer 1 l th -graders participated in the survey, those who did were less 
likely to skip items than 9 th - and 10 th -graders. Six hundred ninety students skipped half or more of 
the survey items and were dropped from further analyses, leaving 10,137 students. In addition, 
1,059 students exhibited questionable response sets, i.e., they responded with the same answer to 
ah items in a block and at least one of the items in the block was negatively worded. These students 
were also dropped from the analytic dataset. After excluding students who skipped over half the 
items and who exhibited questionable response sets, we imputed a complete dataset using multiple 
imputation. All analyses presented below were conducted using this imputed dataset (n=9,078). 
This represents about 60% of ah 9 th - to 1 l th -graders in the district. 



Table D-l: Descriptive Statistics of Demographic Variables 



Full dataset 

Reduced dataset 

Population 



Percent 


Percent 


Percent 


N 

of pop. 

N 

of pop. 

N 

of pop. 

Total students 

10,827 

59.3 

9,078 

49.7 

18,266 


Number of schools 
Gender 

13 


13 


13 


Female 

5,440 

50.3 

4,781 

52.7 

9,077 

49.7 

Male 

5,371 

49.7 

4,287 

47.3 

9,189 

50.3 

Missing 

Race/ethnicity 

16 

0.1 

10 

0.1 



Hispanic 

6,675 

61.6 

5,730 

63.2 

10,859 

59.4 

African American 

2,253 

20.8 

1,701 

18.8 

4,172 

22.8 

White 

1,483 

13.7 

1,284 

14.2 

2,556 

14.0 

Other 

418 

3.9 

353 

3.9 

679 

3.7 

Missing 

16 

0.1 

10 

0.1 



Grade 







9 

3,996 

37.0 

3,378 

37.3 

5,772 

31.6 

10 

3,307 

30.6 

2,779 

30.6 

4,691 

25.7 

11 

3,122 

28.9 

2,598 

28.7 

4,328 

23.7 

12 

385 

3.6 

312 

3.4 

3,517 

19.3 

Missing 

Economically 

disadvantaged 

17 

0.2 

11 

0.1 



Yes 

7,698 

71.1 

6,420 

70.7 

11,158 

60.9 

Missing 

Limited English 
Proficient 

3,129 

28.9 

2,658 

29.3 



Yes/Current 

721 

6.7 

558 

6.1 

1,123 

6.1 

First year out 

172 

1.6 

151 

1.7 



Second year out 

220 

2.0 

192 

2.1 



Missing 

9,714 

89.7 

8,177 

90.1 




Note: Population numbers come from 201 1-12, with the exception of economically disadvantaged 
status, which comes from 2010-1 1 . 


Scale Descriptions 

The student survey was designed with several a priori scales, many of them used in previous 
research projects. Factor analyses on principal components were performed separately on each of 
the proposed survey scales. The student survey included scales measuring the following intended 
constructs: academic engagement, personalization, parent press toward academic achievement, peer 
support for academic achievement, student sense of belonging, student study habits, how well the 
school prepared students for successful futures, school climate, disaster preparation, the degree to 
which students felt expectations for academic press, the degree to which students felt challenged 
academically, the degree to which students showed responsibility through class participation, the 
degree to which the school culture fostered responsible behavior, school safety, bullying, and 
parent connections. Tables D-2 through D-6 present information on each scale, including the items 
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that make up the scale. 


Table D-2: Reliability of Scales for Rigorous and Aligned Curriculum 


Scales 

Factor 

loadings 

Scale 

reliability 

Academic Press: Expectations 


0.67 

My classes really make me think. 

0.609 


My teachers expect me to do my best all the time. 

0.844 


My teachers expect everyone to work hard. 

0.833 


Academic Press: Challenge 


0.73 

In class, how often do you find the work difficult? 

0.746 


In class, how often does the teacher ask difficult questions on tests? 

0.780 


In class, how often does the teacher ask difficult questions? 

0.759 


In class, how often do you have to work hard to do well? 

0.610 


In class, how often are you challenged? 

0.567 


School-Wide Future Orientation 


0.80 

High school is seen as preparation for the future. 

0.591 


All students are encouraged to go to college. 

0.608 


Teachers work hard to make sure all students are learning. 

My teachers make sure that all students are planning for life after 

0.775 


graduation. 

0.747 


My teachers pay attention to all students, not just the top students. 

0.742 


My teachers work hard to make sure that all students stay in school. 

0.793 




Table D-3: Reliability of Scales for Personalized Learning Connections 


Scales 

Factor 

loadings 

Scale 

reliability 

Personalization 


0.88 

How many adults in your school are willing to give extra help with 
your homework if needed? 

0.773 


How many adults in your school are willing to help you solve 
personal problems? 

0.799 


How many adults in your school really care about how you are doing 
in school? 

0.845 


How many adults in your school help you think about what you need 
to do to prepare for college? 

0.852 


How many adults in your school have helped you think about 
whether you are meetings the requirements for graduation? 

0.819 


Student Sense of Belonging 


0.71 

People at this school are like family to me 

0.683 


1 fit in with the students at this school 

0.643 


People at this school care if I am absent 

0.665 


There are people at this school that I can talk to about personal 
matters 

0.692 


1 participate in a lot of activities at this school 

0.509 


There are people at this school that will help me if I need it 

0.661 
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Table D-4: Reliability of Scales for Culture of Learning and Professional Behavior-Academic 
Culture 



Factor 

Scale 

Scales 

loadings 

reliability 

Academic Engagement 


0.73 

1 am usually bored in class 

0.690 


Sometimes 1 get so interested in my work that I don't want to stop 

0.674 


The topics we are studying in this school are interesting 

0.750 


1 usually look forward to my classes 

0.767 


1 work hard to do my best in my classes 

0.533 


1 often count the minutes until class ends 

0.526 


Peer Support for Academic Achievement 


0.80 

My friends try hard in school 

0.669 


My friends and I talk about what we did in class 

0.650 


My friends and I help each other prepare for tests 

0.727 


My friends think it is important to do well in school 

0.751 


My friends and I help each other with homework assignments 

0.746 


My friends think it is important to attend every class 
Student Study Habits 

0.721 

0.72 

1 always study for tests 

0.749 


1 set aside time to do my homework and study 

0.816 


1 try to do well on my schoolwork even if it isn't interesting to me 

0.666 


If I need to study, I don't go out with my friends 
Student Responsibility: Participation 

0.745 

0.82 

How many students in your class attend class regularly? 

How many students in your class come to class prepared with 

0.658 


appropriate supplies and books? 

0.793 


How many students in your class regularly pay attention in class? 

0.816 


How many students in your class actively participate in class activities? 

0.730 


How many students in your class always turn in their homework? 

0.758 


School Climate 


0.74 

My classes prepare me to be successful in college 

0.630 


My teachers encourage me to share my ideas and opinions 

0.633 


1 feel other students at my school respect me 

0.449 


The principal at our school is a good leader 

0.549 


1 receive recognition for doing good work 

0.633 


My teachers treat me with respect 

0.757 


My teachers treat other students with respect 

0.751 





Table D-5: Reliability of Scales for Culture of Learning and Professional Behavior-School Safety 


Scales 

Factor 

loadings 

Scale 

reliability 

Student Responsibility: School Culture 

How many students in your class feel OK to make racist or sexist 


0.73 

remarks? 

0.905 


How many students in your class feel OK to cheat? 

0.853 


School Safety 


0.85 

How safe do you feel outside around the school? 

0.820 


How safe do you feel traveling between home and school? 

0.755 


How safe do you feel in the hallways and bathrooms of the school? 

0.853 


How safe do you feel in your classes? 

0.811 


Bullying/ School Safety 


0.77 

1 know of at least one student who has been bullied electronically 
1 know of at least one person at my school who has been bullied 

0.803 


physically 

0.844 


1 know of at least one person at my school who has been bullied verbally 

0.851 



Table D-6: Reliability of Scales for Connections to External Communities 

Scales 

Parent Press toward Academic Achievement 

To what extent do your parents talk to you about how you are doing 
in your classes? 

To what extent do your parents talk to you about what you are 
studying in class? 

To what extent do your parents talk to you about your homework 
assignments? 

Parent Connections: Involvement 

To what extent are your parents involved in working with you on 
homework or other school projects? 

To what extent are your parents involved in volunteering at your 
school? 

To what extent are your parents involved in helping you decide what 
classes to take? 

To what extent are your parents involved in attending school 
activities or meetings? 

My parents know most of my teachers 

To what extent are your parents involved in communicating with 
your teachers? 

Parent Connections: Future Orientation 

To what extent are your parents involved in encouraging you in 
school? 

To what extent do your parents talk to you about college? 

To what extent do your parents talk to you about future career plans? 


Factor Scale 
loadings reliability 
084 

0854 

0892 

0876 

080 

0.595 

0713 

0565 

0.746 

0.694 

0.742 

0.83 

0.782 

0.873 

0.862 


Reaching for Rigor 


Research Report I October 2013 121 



Results 


The results on the above scales can be found in the body of the report in the respective sections that 
detail findings for each essential component. In addition to the above scales, the survey asked 
students about their participation in various school programs, their educational aspirations, and the 
school personnel with whom they discussed various issues. Some of these findings are included in 
the detailed findings on each essential component, but we present the full results in Tables D-7 
through D-16. The most common school programs in which students across the case study schools 
participated were tutoring, sports, and activities or clubs. Students in Lakeside were most likely to 
attend tutoring, which is not surprising given the school’s emphasis on Learning Time. Students in 
Riverview were more likely to participate in community service while those in Mountainside were 
more likely to participate in credit recovery programs. Students in Valley were least likely to attend 
preparation for the SAT, ACT, or PSAT. These findings are consistent with what emerged in the 
fieldwork. Students in Lakeside also appeared less likely to participate in sports. Students in 
Mountainside and Riverview were more likely to participate in school leadership, perhaps 
indicating a stronger presence of a student government structure in those schools. 


Table D-7. Student Participation in School Programs 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Activities or Clubs 

30* 

23*** 

26*** 

34 *** 

45% 

Tutoring 

58*** 

43* 

75 *** 

26*** 

44 

Sports 

42* 

41 

35*** 

41* 

40 

PSAT, SAT, ACT Prep 

28*** 

12 *** 

37 *** 

34 *** 

25 

Community Service 

18** 

10 ** 

13** 

24** 

16 

School Leadership 

12 *** 

5 *** 

7 ** 

13*** 

8 

Other Test Prep 

9 *** 

5 *** 

11 *** 

7 * 

7 

Credit Recovery 

12 *** 

g** 

8 

5 *** 

7 

ELL Program 

9 *** 

g*** 

7 *** 

2 *** 

4 

Internship 

2 

i 

2* 

1 ** 

1 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s mean rating compared to the mean from 
the other schools combined. 


Riverview had students with greater educational aspirations than the other three case study schools. 
This is consistent with the fieldwork as Riverview had a vision as a college preparatory school. 
Almost three-quarters of students at the school planned to complete a four-year degree or higher. In 
the other three schools, that response ranged from 55% to 68 %. 



Table D-8. Educational Aspirations by School 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Not graduate from HS 

0.3%** 

0 . 8 %** 

0.9%*** 

0.5%*** 

0.9% 

Graduate from HS 

14** 

19 

15 

11 *** 

13 

Pursue a cert, or license 

5 

g*** 

9 

5 *** 

7 

Complete 2-yr college degree 

1 1 ** 

15*** 

17*** 

8 *** 

14 

Complete 4-yr college degree 

46 

3g*** 

40*** 

42*** 

39 

Complete advanced degree 

24*** 

18*** 

19* 

22 *** 

23 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students were most likely to talk to their counselor about what courses to take, with about two- 
thirds of students doing so. There were not large differences between schools in the adults students 
talked to about what courses to take. 

Table D-9. Percentage of Students by School Who Talked to Various Adults about What Courses to 
Take 



LVA Schools 

HVA Schools 

District mean 

Mountainside 

Valley 

Lakeside 

Riverview 

Administrator 

7 %*** 

3% 

3% 

3%*** 

6 % 

Counselor 

63*** 

65*** 

68 

59** 

68 

Teacher 

28 

30 

29 

22 *** 

29 

Coach 

10 *** 

9 *** 

g** 

8 

7 

Other 

14*** 

10 * 

14*** 

12 

12 

No one 

13 

16** 

13* 

15* 

14 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students talked to fewer school personnel about what courses are required for college. There were 
differences in the role of school personnel in helping students know college requirements. In 
Mountainside, only 18% of students did not talk to anyone about course requirements for college, 
indicating that 82% of students did discuss the topic with someone at the school. In Riverview, 
however, 36% of students did not talk to anyone in the school about what courses to take for 
college. This may have been due to the strong role Riverside parents took in helping students 
navigate the path to college. 
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Table D-10: Percen tage of Students by School Who Talked to Various Adults about What Courses 
You Need for College 


LVA Schools HVA Schools 

District 



Mountainside 

Valley 

Lakeside 

Riverview 

mean 

Administrator 

5%*** 

2 %*** 

3%* 

5 %*** 

4% 

Counselor 

53*** 

42 *** 

49*** 

41*** 

44 

Teacher 

32*** 

29 ** 

32*** 

27 * 

45 

Coach 

g*** 

g*** 

3* 

3 

3 

Other 

24*** 

20 *** 


1 1 *** 

11 

No one 

2 g*** 

32 

24*** 

36*** 

30 

* for p<.05, ** 

for p<.01, and *** for p<.001. Statistical significance was calculated based on 


mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students were most likely to talk to teachers, followed by counselors, about how to prepare for 
college admissions tests such as the PSAT, SAT, and ACT. Students in Valley were least likely to 
talk to anyone in the school about these college admissions tests, consistent with the finding above 
that they were the least likely to participate in programs to prepare for them. 


Table D-ll. Percentage of Students by School Who Talked to Various Adults about Preparing for 
the PSAT, SAT, or ACT 


LVA Schools HVA Schools 

District 

Mountainside Valley Lakeside Riverview mean 


Administrator 

5%* 

2 %*** 

4% 

8 %*** 

4% 

Counselor 

36*** 

21 

32*** 

lg*** 

20 

Teacher 

45** 

41*** 

50** 

59 *** 

48 

Coach 

4*** 

5 *** 

2 *** 

1 

2 

Other 

23*** 

g*** 

25 *** 

9 

9 

No one 

29*** 

3g*** 

29*** 

25 *** 

31 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students were most likely to talk to their teachers about how to prepare for standardized tests, with 
about 55-60% of students doing so. There were few large differences between schools. 



Table D-12. Percentage of Students by School Who Talked to Various Adults about Preparing for 
other Standardized Tests 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Administrator 

6 %*** 


4% 

5%*** 

3% 

Counselor 


13* 

lg*** 

io*** 

13% 

Teacher 

55* 

56 

58 

^ 0 *** 

57 

Coach 

4 *** 

2 

2 

1 ** 

2 

Other 

io*** 

10 *** 

11 *** 

8 

8 

No one 

20 *** 

27 

2 i*** 

26 

26 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students were most likely to talk first to a counselor and then to a teacher about filling out college 
or financial aid applications. However, one-third to over half of the students in our case study 
schools did not talk to anyone in school about things. It is worth noting that the data are primarily 
from students in grades 9 to 1 1 who may not have been actively completing these applications yet. 


Table D-13. Percen tage of Students by School Who Talked to Various Adults about Filling Out 
College or Financial Aid Applications 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Administrator 

4 %*** 

2 % 

2 % 

2 %** 

2 

Counselor 

34 *** 

2 i*** 

24*** 

19*** 

24 

Teacher 

20 

18 

2 i** 

15 *** 

19 

Coach 

g*** 

3 * 

2 * 

3 

3 

Other 

15** 

12 

19*** 

13 

12 

No one 

22 *** 

51** 

27*** 

57 *** 

49 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Most students did not talk to any adult at school about how to get a job. The most common adult 
they did discuss this with was a teacher. Students in Mountainside were more likely than students 
in other schools to talk to a coach. 
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Table D-14. Percentage of Students by School Who Talked to Various Adults about How to Get a 
Job 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Administrator 

3%*** 

1 %** 

2 % 

2 % 

2 % 

Counselor 

lg*** 

10 

12 *** 

7 *** 

10 

Teacher 

25 

24 

29*** 

16*** 

24 

Coach 

g*** 

4 

4 


4 

Other 

22 * 

21 

22 * 

20 

21 

No one 

36*** 

47* 

43 *** 

57 *** 

48 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Students were most likely to talk to a teacher, followed by their counselor, about what they wanted 
to do after graduation. Students in Mountainside were more likely than students in other schools to 
talk to a coach. 

Table D-15. Percen tage of Students by School Who Talked to Various Adults about What You Want 
to Do After You Graduate 



LVA Schools 
Mountainside Valley 

HVA Schools 
Lakeside Riverview 

District 

mean 

Administrator 

9%*** 

3%*** 


3%*** 

5% 

Counselor 

37 *** 

24*** 

23*** 

22 *** 

29 

Teacher 

39 

38 

41*** 

33 *** 

38 

Coach 

^3*** 

9 *** 

7 

8 

8 

Other 

26*** 

20 *** 

26*** 

24*** 

22 

No one 

17*** 

29*** 

2 i*** 

33 *** 

26 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 


Most students did not talk to any adult at school about personal or family issues or problems. The 
most common adult they did talk to about getting a job was a teacher or counselor. Students in 
Mountainside were more likely than students in other schools to talk to a coach about personal 


issues. 



Table D-16. Percen tage of Students by School Who Talked to Various Adults about Personal or 
Family Issues or Problems 



LVA 

Mountainside Valley 

HVA 

Lakeside Riverview District mean 

Administrator 

4%*** 

1%*** 

1%*** 

2%*** 

2% 

Counselor 

20*** 

13 *** 

lg*** 

12*** 

15 

Teacher 

22*** 

18 

2i*** 

12*** 

17 

Coach 


7 

5*** 

6 

6 

Other 

15** 

15*** 

2i*** 

17 

17 

No one 

41 *** 

55* 

47*** 

50*** 

53 


* for p<.05, ** for p<.01, and *** for p<.001. Statistical significance was calculated based on 
mean comparisons tests between each case study school’s value compared to the average from the 
district’s other 12 schools. 
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Appendix E: Teacher Survey Data 


Data Description 

Similar to the student survey, we also collaborated with the district to obtain teacher survey data. 
Specifically, we coordinated the survey items we identified as important to this study (guided by 
our research questions and framework of essential components). Many of these survey items were 
incorporated into the district-administered survey to each stakeholder. The surveys were 
administered across all schools in the district, not just in the four case study high schools that were 
the focus of this study. 

The teacher survey was administered through an online portal to all teachers in December 2011 to 
January 2012. The overall response rate was 44% across all high schools, for a total of 577 
respondents. Response rates within case study schools varied considerably, ranging from 30% to 
over 60% of teachers. See Table E-l for demographic characteristics of the teacher survey sample. 

We attempted to conduct a missing data analysis to investigate patterns of item completion. The 
data provided by the district had no missing data. All item had an “N/A” or “I don’t know” option 
and it appeared that the online survey defaulted responses to these options, making it impossible to 
distinguish between a skipped item and a true “N/A” or “I don’t know” response. To conduct a 
missing data analysis, we treated all “N/A” and “I don’t know” responses as missing. Across all 
items, the amount of “missingness” per item ranged from less than 1% of the respondents to 26%. 
There was not an identifiable pattern in the items that were left as “N/A” or “I don’t know”. On 
average, teachers skipped about 5% of the items (range 0-66%). Only one teacher skipped more 
than 50% of the items; this teacher was dropped. In addition, 20 teachers exhibited questionable 
response sets, i.e., they responded with the same answer to all items in a block. These teachers 
were also dropped from the analytic dataset. All analyses presented below were conducted using 
this reduced dataset (n=556). 



Table E-l: Descriptive Statistics of Demographic Variables on Teacher Survey 

Full Dataset Reduced Dataset 


N % of Pop. N % of Pop. 


Total 

577 

44.0 

556 

42.4 



%of 


% of 

Type of Classes 

N 

Teachers 

N 

Teachers 

Advanced Placement /Pre-AP/International 

165 

28.6 

161 

29.0 

Baccalaureate 





Gifted and Talented /Honors 

148 

25.6 

145 

26.1 

Regular/Standard 

468 

81.1 

451 

81.1 

Special Education 

80 

13.9 

79 

14.2 

Remedial 

29 

5.0 

29 

5.2 

Dual-Language/ESL/ELL 

55 

9.5 

55 

9.9 



%of 


%of 

Academic or Leadership Positions 

N 

Teachers 

N 

Teachers 

Department Lead/Chair 

63 

10.9 

62 

11.2 

Grade Level Head/Chair 

13 

2.3 

13 

2.3 

Content Coach/Lead Content Teacher 

44 

7.6 

41 

7.3 

Serve on a school-wide committee or task force 

139 

24.1 

137 

24.6 

Serve on a district-wide committee or task force 

29 

5.0 

29 

5.2 

None of above 

365 

63.3 

349 

62.8 


Note: The total population of teachers in the district’s high schools is 1,310 teachers. 
Administrative data are not available for the type of classes teachers taught and academic or 
leadership positions held. Teachers could select more than one response option for the type of 
classes they taught and academic or leadership positions. 


Scale Descriptions 

The teacher survey was designed with several a priori scales, many of them used in previous 
research projects. Coordinating with the district meant that not all items making up the scale were 
included on the survey and additional items were added. Principal components factor analyses were 
performed separately on each of the proposed survey scales and on the items added by the district. 
The teacher survey included scales measuring the following intended constructs: bullying, data use, 
efficacy, instructional program coherence, personalization-social, school leader instructional 
support, teacher-principal trust, teacher-teacher trust, support for quality instruction, systemic 
performance accountability, supportive and shared leadership, expectations for postsecondary 
education, personalization-school action, teacher accountability, teacher outreach to parents, 
teacher-parent trust, and time to collaborate. Tables E-2 through E-10 present information on each 
scale, including the items that make up the scale. Note that in the body of the report, we do not 
report findings for any scale that had a reliability below 0.6 (i.e., Parent Outreach-Teacher, 
Teacher-Parent Trust, and Career Readiness). 
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Table E-2: Reliability of Scales for Learning-centered Leadership 


Factor Scale 
loadings reliability 


Instructional Program Coherence 0.68 

We have so many different programs in this school that I can't keep -.337 
track of them all. 

You can see real continuity from one program to another in this .809 

school. 

Curriculum, instruction, and learning materials are well coordinated .880 
across the different grade levels at this school. 

There is consistency in curriculum, instruction, and learning materials .809 
among teachers in the same grade level at this school. 

Teacher-Principal Trust 0-90 

My principal is very visible on campus. .827 

Overall, I have confidence in the administrators at my school. .891 

My principal appreciates diversity and respects the cultural .843 

backgrounds of the staff. 

I can reach my principal when I need to. .848 

My school administrators care about me as a person. .788 

Principal Effectiveness at Supporting Quality Instruction 0-92 

Allocates instructional resources to maximize the quality of .835 

instruction. 

Implements procedures to protect instructional time. .818 

Uses the most effective teachers to instruct students at risk of failure. .810 
Discusses instructional practices during faculty meetings. .785 

Monitors the quality of instruction for students at risk of failure. .853 

Utilizes technology to improve teaching and learning. .777 

My school administrators encourage my professional growth and .722 

development. 

My school administrators receive input from staff concerning .648 

planning and revisions to student scheduling. 

My school administrators provide staff with sufficient advanced .676 

notice of upcoming meetings and activities. 



Table E-3: Reliability of Scales for Rigorous and Aligned Curriculum 



Factor 

Scale 


loadings 

reliability 

Postsecondary Expectations 


0.79 

I expect most students in this school to either go to college or attend 

.901 


postsecondary Paining. 

Most of the students in this school are planning to either go to college 

.904 


or attend postsecondary training. 



Career Readiness 


0.55 

I help students plan for college/career-readiness outside of class 

.818 


time. 



I feel that it is part of my job to prepare students to succeed in 

.841 


college. 




Table E-4: Reliability of Scales for Personalized Learning Connections 

Factor Scale 

loadings reliability 


Personalization-Extra Help 0.66 

Extra help from you .641 

Extra help from other staff members during regular school hours .578 

Extra help from school staff outside of regular school hours .597 

I help my students with school work outside of class time. .604 

Mostly when I contact parents, it's about problems or trouble. -.539 

Personalization-Structural Support 0.76 

Parent-teacher meetings to discuss what the school and the student's .745 
parents/guardians can do to help 

Help in choosing their classes .826 

Referrals to community organizations for assistance .798 

Personalization-Social 0-77 

Know their academic aspirations .732 

Know their academic background prior to this year .699 

Know their home life .820 

Know who their friends are .778 

Spend time with them through extra-curricular activities .615 
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Table E-5: Reliability of Scales for Culture of Learning and Professional Behavior-Academic 
Culture 


Factor Scale 
loadings reliability 


Efficacy 0.68 

If a student did not remember information 1 gave in a previous lesson, 1 .772 

would know how to increase their retention in the next lesson. 

I know some techniques to redirect disruptive students quickly. .822 

I can get through to even the most difficult or unmotivated students. .763 

Teacher-Teacher Trust 0-63 

The teachers on my campus respect my ability to provide quality .814 

instruction. 

Other teachers on my campus respect diverse professional opinions and .829 

practices at my school. 

This school is a safe place to work. .598 

I have at least one colleague in whom I can confide at my campus. .500 

Positive Faculty Culture 0.83 

My school administrators respect diverse professional opinions at my .813 

school. 

My school administrators use data to identify areas in need of .635 

improvement. 

My school administrators receive input from staff concerning campus .734 

performance objectives. 

I enjoy working at this school. .724 

At my campus, 1 have the opportunity to do what 1 do best every day. .709 

I regularly receive recognition/praise for doing good work. .7 10 

The District has clearly emphasized strategies designed to eliminate the .588 

achievement gap between white and minority students. 




Table E-6: Reliability of Scales for Culture of Learning and Professional Behavior-Collaboration 
and Feedback 


Factor Scale 
loadings reliability 


Frequency of Collaboration: How often have you: 0.7 1 

Created lesson plans with other teachers .759 

Reviewed student work with other teachers .622 

Reviewed student data with other teachers .670 

I have a planning period at the same time as most of the other teachers in .616 
my department/grade level. 

Frequency of Peer Observation and Feedback: How often have you: 0.84 

Observed other teachers while they teach .752 

Received feedback from other teachers regarding how you teach your .706 

students 

Provided feedback to other teachers based on your observations of their .796 

teaching 

Co-taught with or modeled a lesson for another teacher .709 

Worked to identify individual students' learning needs with other teachers .572 
Collaboration Improves Instruction: To what extent have these activities 0.94 

contributed to improving your instruction? 

Created lesson plans with other teachers .664 

Reviewed student work with other teachers .730 

Reviewed student data with other teachers .709 

Observed other teachers while they teach .769 

Received feedback from other teachers on how you teach your students .812 

Provided feedback to other teachers based on your observations of their .780 

teaching 

Co-taught with or modeled a lesson for another teacher .635 

Worked to identify individual students' learning needs with other teachers .767 


Table E-7: Reliability of Scales for Culture of Learning and Professional Behavior-School Safety 



Factor 

loadings 

Scale 

reliability 

Bullying 


0.85 

I know of students who have been physically bullied at my school. 

.906 


I know of students who have been verbally bullied at my school. 

.911 


I know of students at my school who have been bullied on the 
Internet. 

.819 
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Table E-8: Reliability of Scales for Connections to External Communities 


Factor Scale 

loadings reliability 

Parent Outreach-Teacher 0-43 

Involving parents is my responsibility as a teacher. .841 

I work closely with my students' parents to meet my students' needs. .725 

Parent Outreach-School Support 0-61 

My principal pushes teachers to communicate regularly with parents. .604 
Parents are greeted warmly when they call or visit the school. .800 

This school regularly communicates with parents about how they can .825 
help their children learn. 

Teacher-Parent Trust 0.31 

Most parents know how to help their children with school work at .817 

home. 

Parent involvement can increase my classroom effectiveness. -.030 

I do not have the time to involve parents in very useful ways. -.092 

Most parents give timely responses to my requests. .817 


Table E-9: Reliability of Scales for Systemic Use of Data 

Factor Scale 

loadings reliability 

Data use 0.88 

CBA data are used to modify/enhance instruction in my classroom. .422 

1 use multiple sources of student data (i.e., test data, interventions, .691 

attendance) to modify/enhance instruction in my classroom. 

1 use data to plan lessons. .858 

1 use data to set learning goals. .865 

1 use data to form small groups of students for targeted instruction. .795 

1 use data to develop recommendations for tutoring or similar .822 

interventions. 

1 use data to identify students' responses to interventions that they .852 

have received. 

The requests I receive for data are reasonable. .639 



Table E-10: Reliability of Scales for Systemic Performance Accountability 


Factor Scale 
loadings reliability 


Instructional Support from School Leader 0-92 

I discussed my teaching with my principal or an assistant principal. .789 

My principal or an assistant principal observed my teaching for at .775 

least 10 minutes. 

My principal or an assistant principal provided me with feedback to .871 
improve my instruction after observing my teaching. 

My principal or an assistant principal reviewed my students' work .809 

with me. 

My school administrators provide me with ongoing feedback and .793 

guidance. 

My school administrators have spoken with me about my progress so .811 
far this year. 

My principal/assistant principal/evaluator considered my individual .856 
professional evaluation results when recommending professional 
development activities to me. 

Teacher Accountability 0-74 

My PDAS teacher evaluation provides valuable feedback that .727 

improves my teaching. 

The teachers on my campus are committed to providing quality .560 

instruction for our students. 

There are consequences for teachers who do not perform well. .770 

Other teachers hold me accountable for my performance. .728 

Teachers who don't perform well are given opportunities to improve. .687 

Principal Effectiveness at Systematic Performance Accountability 0.88 

Implements programs and practices to hold faculty accountable to .904 

reach the highest levels of performance. 

Advocates that all students are accountable for achieving high levels .886 
of performance in both academic and social learning. 

Challenges faculty who do not hold all students accountable for .908 

achieving. 
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Appendix F: District Perspectives on High School Effectiveness 


Interviews conducted with central office personnel provide insight into the district as the unit of work. 
Despite a focus in the interviews on similarities and differences among the four schools in the study, few 
concrete findings emerged in this area. The clear exception to this conclusion was Riverview, which 
central office personnel regularly identified as an outlier, because of its relative high concentrations of 
higher SES students compared to the other three schools. 

Interview data did provide the impression that the two higher value-added (HVA) schools were marked 
by some conditions that were stronger or more fully developed than in the two lower value-added (LVA) 
schools. To begin with, central office personnel reported sponger administration/leadership in the HVA 
schools, although the issue of cause and effect was never addressed directly. Some personnel also noted 
that strong leadership was responsible for the recent increase in the achievement at Valley, one of the 
LVA schools. Second, district personnel indicated that teachers in HVA schools internalized 
accountability for student learning to a greater extent than their peers in LVA schools. Along with this, 
district personnel suggested that the HVA schools had strong culture and organizational structures in 
place, such as tutoring at Lakeside and the culture of teacher leadership/responsibility at both Lakeside 
and Riverview. Additionally, central office personnel defined “academic press” in part by the percentage 
of students taking Advancement Placement classes. Their perspective was that HVA schools were 
exercising this approach with greater vigor and success than were LVA schools. 

The focus of the central office interviews was on the four schools in the study, especially distinctions 
among them. At the same time, however, interviews provided additional insights about district office 
operations, especially what was valued at that level of the school system. There is little doubt that student 
academic success on state tests was the sine quo non of effectiveness for all schools. Third, there was 
pressure to have schools become more meaningful and relevant for students. This effort was grounded in 
the logic that meaningfulness would improve both the schools’ holding power and their student learning 
outcomes. Pathways to accomplish this goal included a focus on student-centered curriculum and student 
(and parent) choice in the selection of varied and distinct courses of study (e.g., Gold Seal Programs of 
Choice). 
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